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ABSTRACT 


"Adventures  in  Assessment”  is  an  annual  journal  of  the 
system  for  Adult  Basic  Education  Support  (SABES) , a comprehensive  training 
and  technical  assistance  initiative  for  adult  literacy  educators  and 
programs.  SABES  accomplishes  this  goal  through  staff  and  program  development 
workshops,  consultation,  mini -courses,  mentoring,  peer  coaching,  and  other 
training  activities  provided  by  five  Regional  Support  Centers  located  at 
community  colleges  throughout  Massachusetts.  The  first  three  volumes  present 
a comprehensive  view  of  the  state  of  practice  in  Massachusetts  through 
articles  written  by  adult  literacy  practitioners.  Volume  1,  "Getting 
Started,"  includes  start-up  and  intake  activities;  volume  2,  "Ongoing," 
shares  tools  for  ongoing  assessment  as  part  of  the  learning  process;  volume 
3,  "Looking  Back,  Starting  Again,"  focuses  on  tools  and  procedures  used  at 
the  end  of  a cycle  or  term,  including  self,  class,  and  group  evaluation  by 
both  teachers  and  learners.  Volume  4 covers  a range  of  interests,  and  volume 
5,  "The  Tale  of  the  Tools, " is  dedicated  to  reflecting  on  Component  3 tools 
of  alternative  assessment.  Volume  6,  "Responding  to  the  Dream  Conference,"  is 
dedicated  to  responses  to  volumes  1-5.  Volume  7,  "The  Partnership  Project," 
highlights  writings  from  a memtoring  project  for  practitioners  intersted  in 
learning  about  participatory  assessment.  Volumes  8-11  cover  a range  of 
topics,  including  educational  reform,  workplace  education,  learner 
involvement  in  asseessment,  etc.  (MN) 


Reproductions  supplied  by  EDRS  are  the  best  that  can  be  made 
from  the  original  document. 
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SABES  is  the  System  for  Adult  Basic  Education  Support,  a comprehensive  training  and  technical 
assistance  initiative  for  adult  literacy  educators  and  programs.  Its  goal  is  to  strengthen  and  enhance 
literacy  services  and  thus  to  enable  adult  learners  to  attain  literacy  skills.  SABES  accomplishes  this 
goal  through  staff  and  program  development  workshops,  consultation,  mini-courses,  mentoring  and 
peer  coaching,  and  other  training  activities  provided  by  five  Regional  Support  Centers  located  at 
community  colleges  throughout  Massachusetts . 

SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect,  evaluate,  and  disseminate  ABE 
materials,  curricula,  methodologies,  and  program  models . Each  of  the  five  SABES  Regional  Support 
Centers  similarly  offers  program  support  and  a lending  library . SABES  maintains  an  Adult  Literacy 
Hotline,  a statewide  referral  service  which  responds  to  calls  from  new  learners  and  volunteers . The 
Hotline  number  is  1-800-447-8844. 

The  SABES  Central  Resource  Center,  located  at  World  Education,  publishes  a statewide  quarterly 
newsletter,  ** Bright  Ideas, ' and  journals  on  topics  of  interest  to  aduli  literacy  professionals,  such  as 
this  volume  of  “Adventures  in  Assessment . " 

The  first  three  volumes  of  “Adventures  in  Assessment”  present  a comprehensive  view  of  the  state  of 
practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners.  Volume  1,  Getting 
Started,  included  start-up  and  intake  activities.  Volume  2,  Ongoing,  shares  tools  for  ongoing 
assessment  as  part  of  the  learning  process ; Volume  3,  Looking  Back,  Starting  Again,  focused  on  tools 
and  procedures  used  at  the  end  of  a cycle  or  term,  including  self  class,  and  group  evaluation  by  both 
teachers  and  learners.  Volume  4 covered  a range  of  interests.  Volume  5,  Tale  of  the  Tools  was 
dedicated  to  reflecting  on  component  3 tools  of  alternative  assessment.  Volume  6,  Responding  to  the 
Dream  Conference,  was  dedicated  to  responses  to  Volumes  1-5.  Volume  7,  The  Partnership  Project, 
highlighted  writings  from  a mentoring  project  for  practitioners  interested  in  learning  about 
participatory  assessment.  Volume  8 covered  a range  of  topics  from  education  reform  to  learner 
involvement  in  assessment.  Volume  9 looked  at  assessment  in  a volunteer  program,  native  language 
literacy,  ESOL  programs,  and  evaluations  in  workplace  education  programs. 

We  would  like  to  see  your  contribution,  especially  if  you  are  doing  native  language  literacy  or  low- 
level  ESOL  literacy.  Contact  editor,  Alison  Simmons,  to  discuss  your  submission. 

Permission  is  granted  to  reproduce  portions  of  this  journal.  We  request  appropriate  credit  be  given 
to  “Adventures  in  Assessment”  and  to  the  author. 

“Adventures  in  Assessment " is  free  to  DOE-funded  Massachusetts  programs;  out-of-state  requests 
will  be  charged  a nominal  fee,  at  cost.  Please  write  to,  or  call: 
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Introduction 


“ Constructing  new  images  of  adults-im- 
ages  built  on  assumptions  of  dignity  and 
competence,  of  literacy  as  reflective  and 
self-critical  practice,  and  of  learning  as 
participatory— requires  that  we  rethink  or 
reconceptualize  not  only  our  notions  of 
what  counts  as  literacy  but  also  our  methods 
of  inquiry— the  processes  we  use  to  docu- 
ment and  assess  learning.  ”l 

Across  Massachusetts,  literacy  programs  are 
struggling  with  a common  concern-learner  assess- 
ment. To  assess  levels  of  literacy  and  English 
language  proficiency  and  to  document  progress, 
programs  likely  use  one  or  more  of  these  methods: 
standardized  testing,  materials-based  assessments, 
competency-based  assessment,  and  participatory 
assessment.2 

The  primary  purpose  of  this  guidebook  is  to 
explore  participatory  assessment  or  learner-cen- 
tered approaches  to  assessment  and  evaluation. 
Participatory  literacy  education  is  “based  on  the 
belief  that  learners,  their  characteristics,  aspira- 
tions, backgrounds,  and  needs  should  be  at  the 
center  of  literacy  education.  ”3  Participatory  assess- 
ment means  a collaborative  relationship  among 
learners  and  program  staff  in  determining  the 
goals,  texts  and  contexts  of  assessment,  as  well  as 
in  judging  its  outcomes.4 

A few  key  principles  guide  alternative  forms  of 
assessment.  The  most  important  principle  is  that 
assessment  must  be  done  with  the  learner  and  not  to 
the  learner.  Second,  it  must  be  integrated  into 
instruction— it  should  inform  curriculum  develop- 
ment which  in  turn  provides  feedback  that  will 
make  programs  more  effective.3 

By  definition,  participatory  assessment  or  learner- 
centered  assessment,  differs  from  program  to  pro- 
gram, teacher  to  teacher,  and  learner  to  learner. 


However,  a tool  kit  of  assessment  procedures  as 
suggested  by  Hemmendinger*  and  Auerbach7  might 
include  activities  around  the  following  three  com- 
ponents: 

- Start-up  or  Intake  Activities 

- Along  the  Way  or  On-going  Activities 

- Looking  Back  or  End  of  Cycle  Activities 

The  first  issue  of  Adventures  in  Assessment 
looks  at  “Start-up”  activities.  Some  of  the  tools 
included  here  aim  to  get  a sense  of  stuaeni  interests 
and  goals  as  they  enter  a program.  As  Auerbach 
says,  “They  provide  base-line  data  about  what 
students  can  already  do  with  language  and  literacy, 
how  they  think  about  it  and  what  they  may  want  to 
do  as  a result  of  instruction.  ”8 

Volume  two  of  Adventures  in  Assessment  will 
focus  primarily  on-going  assessment  activities,  the 
ways  we  document  learning  as  it  takes  place  and  the 
ways  we  integrate  it  into  instruction.  However,  we 
are  open  to  receiving  submissions  on  any  activities 
or  tools  that  you  have  developed  that  may  be 
defined  as  participatory  or  learner-centered.  We 
are  also  interested  in  readers  responding  to  articles, 
especially  if  the  article  is  used  to  experiment  with 
the  ideas  or  tools  described. 

The  first  piece,  “Assessment  Issues”  synthe- 
sizes the  various  articles  I have  read  about  alterna- 
tive assessment.  It  was  first  published  in  1990  as  a 
working  paper  for  S ABES  as  an  attempt  to  put  forth 
for  readers  why  the  topic  of  evaluation  and  assess- 
ment has  become  so  critical  to  the  field  of  adult 
education. 

In  “Partners  in  Evaluation,”  Johan  Uvin,  a 
workplace  ESOL  teacher,  emphasizes  that  evalu- 
ation is  an  integral  part  of  curriculum  development. 
He  describes  the  challenge  of  striking  a balance  be- 
tween evaluation  needs  and  preferences  of  all  who 
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have  vested  interests,  from  learners  to  funders.  He 
believes  evaluation  encompasses  three  levels: 
assessment  of  participants,  evaluation  of  the  pro- 
gram, and  evaluation  of  organizational  change. 

Lucille  Fandel  describes  the  importance  of  un- 
covering themes  and  issues  as  an  integral  part  of 
initial  assessment  in  “Getting  in  Touch.”  She 
relates  how  she  adapted  and  created  tools  that  help 
her  and  her  Latino  students  make  the  link  between 
lesson  content  and  their  goals  for  learning  English. 

Janet  Kelly,  in  her  article  describes  her  own 
“Assessment  Adventures”  and  illustrates  what  as- 
sessment looks  like  in  a whole  language  ABE 
program.  She  uses  a portfolio  approach  to  show 
how  initial  and  early  assessment  information  even- 
tually becomes  a comprehensive  record  of  on- 
going assessment.  In  future  volumes,  Janet  will 
share  her  work  in  on-going  reading  and  writing 
assessment.  Following  Janet’s  piece  is  a related 
article  by  Lindy  Whiton.  Her  “Down  and  Dirty 
Miscue  Analysis,”  elaborates  on  one  of  the  evalu- 
ation tools  Janet  describes. 

Martha  Germanowski  documents  the  devel- 
opment of  “The  Education  Goals  Assessment 
Packet,”  how  it  started  and  how  it  looks  now.  Her 
model  illustrates  how  we  can  use  and  adapt  other 
peoples  ideas  and  tailor  them  to  fit  our  needs.  .Her 
packet  also  illustrates  how  skills  can  be  integrated 
with  goals. 

In  his  introduction  to  “Alternative  Assessment: 
An  Annotated  Bibliography,”  Don  Robishaw  ar- 
gues against  traditional  standardized  quantitative 
approaches  in  favor  of  alternative  approaches  which 
help  practitioners  “set  up  structures  for  improving 
learning  opportunities  and  “inform  the  develop- 
ment of  curriculum  which  is  based  on  the  learners 
evolving  progress.  ” 

While  each  of  these  writings  falls  (loosely) 
under  the  theme  of  start-up  activities,  there  are 
other  lenses  through  which  to  view  them.  The  most 
obvious  is  that  they  are  written  by  adult  basic 


education  and  English  as  a second  language  practi- 
tioners. Though  the  ABE/ESL  lens  may  be  helpful, 
you  might  want  to  also  pay  attention  to  what  a 
whole  language  approach  (Kelly)  and  a participa- 
tory approach  (Fandel  and  Uvin)  have  in  common 
and  how  they  differ.  You  might  want  to  look  at  how 
these  practitioners  have  adapted  others  ideas,  forms 
and  methods  for  their  own  purposes,  and  how  they 
have  made  the  tools  their  own  (Germanowski). 

It  is  our  hope  that  this  guidebook  will  become 
a resource  by  and  for  practictioners  to  select  and 
adapt  tools  for  their  own  contexts.  The  develop- 
ment of  this  resource  depends  upon  program-based 
practitioner  research,  the  results  of  which  will  help 
develop  the  field  of  adult  education. 

—Loren  McGrail,  editor 
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Assessment  Issues: 
Research  and  Practice 


by  Loren  McGrail 


I II  n recent  years  the  subject  of  testing  and  as- 
|§§  sessment  in  adult  literacy  and  education  has 
mmJK  become  a focus  of  growing  concern,  both 
nationally  and  locally.  Decisions  about  the  nature 
and  purpose  of  assessment  are  critically  important  to 
literacy  policy  and  practice  at  all  levels.  Adult 
learners;  teachers,  administrators,  and  funders  all 
agree  that  what  programs  choose  to  assess  and  the 
methods  they  employ  affect  the  quality  of  teaching 
and  learning.  Assessment  in  adult  literacy  is  a central 
issue  with  high  stakes:  “The  authority  vested  in 
these  tests  can  determine  the  way  programs  are  de- 
veloped, what  is  taught,  and  the  climate  of  teaching 
and  learning.  It  shapes  legislation  and  funding 
policies  of  public  and  private  agencies.  It  is  tied  to 
welfare  eligibility  for  young  parents.  It  drives  gov- 
ernment job  training  programs.  It  can  deny  entry  into 
military,  or  crucial  access  to  a diploma  or  a job.” 
(Business  Council  for  Effective  Literacy  Newsletter 
No.  22, January  1990) 

The  Mandate 

In  April  of  1988  Congress  passed  legislation 
which  called  for  the  use  of  standardized  tests  to 
evaluate  all  adult  basic  education  (ABE)  and  English 
as  a second  Language  (ESL)  programs  funded  under 
the  Adult  Education  Act.  These  Adult  Education 
Amendments  (Public  law  100-97)  and  the  imple- 
menting regulations  of  the  U.S.  Department  of 
Education  (August  1989)  require  that  the  results  of 
standardized  tests  be  used  as  one  indicator  of  pro- 


gram effectiveness  and  that  at  least  a third  of  local 
programs  which  apply  for  federal  ABE  funds  (ad- 
ministered in  Massachusetts  by  the  DOE)  should  be 
evaluated  through  the  use  of  standardized  tests,  (see 
BCEL  Newsletter,  No.22,  January  1990) 

While  the  federal  mandate  for  standardized  as- 
sessment in  ABEand  ESL  is  objectionable  to  many, 
it  is  also  a sign  that  adult  education  h as  come  of  age, 
for  the  need  for  accountability  often  accompanies 
the  investment  of  significant  resources. 

The  mandate,  however,  also  highlights  the 
growing  concern  of  literacy  practitioners,  theorists, 
and  test  designers  about  what  standardized  tests 
actually  test  and  for  what  purposes. 

As  standardized  tests  have  come  into  sweeping 
use,  so  have  complaints  about  them  and  their  valid- 
ity. For  many  the  most  important  question  is,  do  the 
test  results  tell  us  anything  of  real  value  either  about 
learner  progress  or  program  effectiveness? 


The  teachers  evaluated  me  to  find  out 
how  much  1 knew  and  compared  me 
with  the  rest  of  the  class.  It  made  me 
feel  small.  —Carmen  Ferino 


The  Debate 

The  use  of  standardized  testing  most  likely  de- 
rives in  part  from  their  relative  ease  of  administration 
and  what  is  viewed  by  some  as  their  reliable  and 
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objective,  quantitative  data  on  the  achievement, 
abilities,  and  skills  of  students-data  that  is  free  from 
individual  judgement.  Because  the  tests  and  the 
conditions  are  (theoretically)  constant,  they  are 
thought  to  be  useful  indicators  of  individual  progress 
over  time  (pre-  and  post-  testing).  Thus,  by  exten- 
sion, they  are  thought  to  be  useful  indicators  of 
program  effectiveness. 

According  to  Susan  Lytle  of  the  University  of 
Pennsylvania,  the  chief  spokesperson  and  advocate 
for  alternative  forms  of  assessments,  these  charges 
of  “quantifiable  and  reliable  data”  for  program 
evaluation  are  questionable  at  best.  For  Lytle, 
(“From  the  Inside  Out:  Reinventing  Assessment”  in 
Focus  on  Basics,  Volume  2,  Number  1 , Fall  1988), 
the  equation  of  learner  progress  assessment-de- 
fined as  increased  reading  levels— with  program  ef- 
fectiveness is  problematic  for  two  reasons:  1)  It  ig- 
nores other  legitimate  criteria  for  evaluating  a liter- 
acy program  like  the  quality  of  the  curriculum,  teach- 
ing, or  its  connection  to  significant  social  issues 
relevant  to  students  lives  and  interests;  and,  2)  it  Ms 
to  recognize  that  increases  in  reading  scores  have 
little  to  do  with  the  way  adults  live  and  use  literacy  in 
the  real  world. 

So,  despite  the  relative  ease  of  administrating 
tests,  few  adult  educators  are  satisfied  with  the 
quality  of  information  revealed  and  many  more  are 
deeply  dissatisfied  with  the  effects  such  testing  has 
had  on  teaching  and  learning.  In  a special  1989  issue 
on  alternative  assessment  from  the  Literacy  Assis- 
tance Center  in  New  York,  this  grievance  was  articu- 
lated clearly:  “There  seems  to  be  considerable 
agreement  among  adult  literacy  practitioners  that  the 
TABE  (whether  it  be  the  1 976  or  1 987  edition)  is  not 
only  an  inappropriate  instrument  for  individual  as- 
sessment, but  it  does  not  inform  the  teaching  and 
learning  process,  and  it  in  fact  may  act  to  discourage 
students  as  they  re-involve  themselves  in  the  educa- 
tional process.  ” 

Other  practitioners  and  theorists  agree  with 


Lytle  and  Johnston  (1986)  that  by  assessing  only 
decontextualized  subskills  like  word  recognition, 
paragraph  comprehension,  and  vocabulary,  we  fail 
to  measure  the  ways  in  which  adult  learners  already 
can  and  do  use  literacy  in  their  daily  lives.  Still  others 
argue  that  because  the  tests  focus  on  product  rather 
than  process,  they  tell  us  very  little  about  the  affec- 
tive or  metacognitive  factors  in  literacy  acquisition. 

Even  Dr.  Thomas  Sticht  in  his  report  to  the  U.S. 
Department  of  Education,  "Testing  and  Assessment 
in  Adult  Basic  Education  and  English  as  a Second 
Language  Programs,"  advocates  that,  “It  may  be  de- 
sirable to  separate  testing  for  program  accountability 
from  testing  for  instructional  decision  making.  ” His 
advice  should  provide  some  guidance  to  literacy  and 
ABE  programs  caught  in  the  eternal  conflict  of  how 
to  be  accountable  to  the  funders  and  the  public  in  a 
quantifiable  way  and  how  to  create  ways  for  teachers 
to  use  assessment  tools  and  processes  that  are  an 
integral  part  of  teaching  and  learning  and  that  can 
inform  curricula. 

The  call  for  alternative  forms  of  assessment  is 
coming  from  all  directions,  including  many  main- 
stream and  professional  organizations  and  educators. 
Elsa  Auerbach  of  the  University  of  Massachusetts  in 
her  book,  Making  Meaning,  Making  Change, 
quotes  the  Board  of  Directors  of  the  International 
Reading  Association's  its  1988  statement:  “Reading 
assessment  must  reflect  recent  advances  in  the 
understanding  of  the  reading  process.  The  IRA  is 
concerned  that  instructional  decisions  are  too  often 
made  from  assessments  which  define  reading  as  a 
sequence  of  discrete  skills  that  students  must  master 
to  become  readers.  Such  assessments  foster  inap- 
propriate instruction.  ” 

Alternative  Assessment 
Approaches/Tools 

How  is  student  assessment  conducted? 

- oral  interview 

- tests  (what  kind?  standardized/ program-developed?) 


- performance  standards  (measure  of  competencies?) 

- collections  of  student  work  (writing  samples?) 

- observation  of  classroom  interactions? 

- self-report 

- on-going  documentation  (teacher  reports?) 

What  counts  as  progress? 

- reading  level  gains? 

- test  scores? 

- affective  gains  in  self-confidence,  etc.. 

- ability  to  use  language/literacy  outside  the  class? 

- ability  to  make  personal,  family,  school,  classroom  and/ 
or  community  changes? 

(Auerbach,  1990) 


A few  key  principles  guide  whai  constitutes  al- 
ternative assessment.  The  first  is  that  it  must  be 
“program-based”  and  “learner-centered”.  The  sec- 
ond is  that  it  should  help  the  learner  achieve  his  or  her 
goals.  In  other  words,  what  is  assessed  must  re- 
flectwhat  the  learner  wishes  or  needs  to  accomplish. 
Third,  the  process  must  build  upon  the  learner’s 
strengths,  not  deficits;  it  must  be  a process  that  is 
done  with  the  learner  and  not  to  the  learner.  Fourth, 
assessment  should  be  part  of  the  learning  experi- 
ence--“an  on-going  collaboration  between  the 
teacher,  the  learner,  and  the  text,  to  review,  and 
refocus  what  should  take  place  in  light  of  progress 
being  made.”  Fifth,  it  should  not  depend  upon  a 
single  procedure  but  a variety  of  procedures.  And 
most  important  it  should  provide  feedback  that  will 
make  programs  more  effective  ( BCEL  Newsletter, 
January,  1990). 

By  definition,  learner-centered  assessment  differs 
from  program  to  program,  teacher  to  teacher,  and 
learner  to  learner.  However,  a tool  kit  of  assessment 
procedures  that  communicates  respect  for  adults,  for 
what  they  bring  with  them  to  the  learning  experience 
and  for  why  they  have  come  might  include  the 
following: 

In-take  or  start-up  activities  to  get  a sense  of 
student  strenghts,  interests,  goals  and  needs.  These 


initial  interviews  can  be  helpful  in  planning  lessons 
as  well  as  providing  data  about  what  students  can 
already  do  with  language  and  literacy  and  how  they 
think  about  it.  They  can  be  done  atthe  beginning  of 
a cycle  (pre-instruction  placement  tool)  or  they  can 
be  integrated  into  instruction.  They  include: 

- informal  interviews 

- reading  samples 

- writing  samples 

- goal-setting  activities. 


Along  the  way  in-class  activities  that  docu- 
ment learning  as  it  takes  place.  These  activities  are 
imcgiaicu  in lu  iustiuCtiGn  on  a regular  basis.  They 
include: 


-journals  kept  by  students 
-journals  kept  by  teachers 
-portfolios  or  writing  folders 
- anecdotes. 


Activities  at  the  end  of  the  cycle  that  reflect 
learning,  teaching,  curriculum  and  program  design. 
They  often  involve  both  teachers  and  learners  to- 
gether and  include: 

- student  self-evaluation  charts  or  check  lists 

- peer  interviews 

- student-teacher  conferences 

- class  or  program  evaluations.  (Hemmendin- 
ger,  1988) 


The  Future:  Directions  and  Implications 

Along  with  learner-centered  or  participatory 
approaches  to  adult  education,  the  alternative  assess- 
ment movement  is  based  on  recent  cross-cultural 
and  ethnographic  research.  This  research  provides 
support  for  the  notion  of  “ many  literacies”  or  “ mul- 
tiple literacies”  —diverse  literacy  practices  where  by 
learners  connect  literacy  to  their  everyday  life  and 
find  ways  to  determine  for  themselves  the  conditions 
under  which  they  will  use  reading  and  writing.  Brian 
Street,  Shirley  Brice  Heath  and  Scribner  and  Cole, 
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by  focusing  on  the  social,  political,  and  economic 
nature  of  these  practices,  tell  us  how  standarized 
tests  don’t  reflect  universal  literacy  but  rather,  “at- 
tribute value  to  particular  literacy  conventions. " 

The  measurement  (standardized  tests)  of  these 
conventions  are  what  funders  and  legislators  use  to 
determine  program  accountability  and  effective- 
ness. Lytle  believes  this  is  due  in  part  to  the  fact  that 
funders  lack  good  information  about  the  qualitative 
effects  of  programs  on  learners’  lives.  She  advo- 
cates two  basic  strategies  to  remedy  this  situation. 
The  first  is  to  invite  wider  participation  in  the  conver- 
sation about  alternative  assessment.  The  second  is  to 
conduct  program-based  practictioner  research  si- 
multaneously across  the  country  to  strengthen  these 
new  conceptual  frameworks  and  to  exchange  and 
critique  innovative  practices.  Auerbach  concurs  with 
Lytle  that  the  cornerstone  of  qualitative  research  is 
documenting  what  happens,  when  it  happens, or  the 
teacher’s  ability  to  know  the  students,  and  to  notice 
and  record  their  development  in  a variety  of  areas. 
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Partners  In  Evaluation1 
Evaluating  the  South  Cove  Manor  Nursing  Home 

Workplace  Education  Program 

with  Participants 


by  Johan  Uvin 


§i§If|;  or  me,  one  of  the  exciting  aspects  of  being 

■|||  a Workplace  ESL  teacher  at  the  South 
jj  Cove  Manor  Nursing  Home  is  that  my  job 
description  is  defined  more  broadly  than  in  more 
traditional  educational  settings.  As  a teacher,  I am  not 
only  responsible  for  instruction  but  for  coordination 
and  curriculum  development  as  well,  with  assess- 
ment and  evaluation  being  an  integral  part  of  the 
latter.  In  this  article,  I take  the  excitement  that  goes 
hand  in  hand  with  the  planning  and  implementation 
of  assessment  and  evaluation  for  granted  and  focus 
mainly  on  the  challenges  that  the  selection  and 
design  of  appropriate  assessment  and  evaluation 
procedures  and  activities  created  for  me  and  how  I 
resolved  them.  It  is  my  intent  that  teachers  with 
similar  role  definitions  and  in  similar  contexts  will 
find  my  experiences  useful  in  their  own  assessment 
and  evaluation  practice. 

Context 

The  South  Cove  Manor  Nursing  Home 
Workplace  Education  Program  was  established  in 
1988  with  funds  from  the  National  Workplace  Liter- 
acy Partnership  Program.  The  goal  is  to  assure  the 
quality  of  patient  care  by  upgrading  the  communica- 
tion skills  of  nursing  assistants  and  entry-level  work- 
ers in  other  departments  of  this  bilingual  nursing 
home  on  the  edge  of  Boston’s  Chinatown.  To 
achieve  this  goal,  the  nursing  home  administration 
and  management  collaborate  with  the  Chinese 
American  Civic  Association  who  act  as  the  learning 


provider.  An  Advisory  Board  consisting  of  repre- 
sentatives of  each  of  the  workplace,  ESL  classes, 
supervisors,  licensed  staff,  teachers  and  managers 
meets  regularly  to  discuss  the  planning,  evaluation, 
and  monitoring  of  the  program  as  a whole.  The 
meetings  of  the  Advisory  Board  provided  the  con- 
text for  the  evaluation  work  that  I describe  in  the 
remainder  of  this  article. 

Challenges 

The  first  challenge  for  me  was  to  strike  a 
balance  between  the  different  evaluation  needs  and 
preferences  expressed  by  all  those  who  had  a vested 
interest  in  the  program:  learners,  teachers,  supervi- 
sors, administrators,  managers,  sponsors,  and  fun- 
ders. It  was  also  important  not  to  ignore  my  own 
beliefs  about  assessment  and  evaluation.  Other  fac- 
tors I needed  take  into  account  were  my  own  skills 
in  assessing  and  evaluating,  the  limited  resources 
that  could  be  allocated  to  assessment  and  evaluation, 
and  the  evaluation  guidelines  as  specified  in  the 
federal  grant. 

Getting  Started 

As  with  other  planning  tasks,  I accomplished 
the  planning  and  implementation  of  assessment  and 
evaluation  as  a collaborative  effort  of  all  those  af- 
fected by  the  program.  I chose  the  meetings  of  the 
Advisory  Board  as  the  forum  to  facilitate  direct 
communication  on  the  issue  between 
learners, teachers,  supervisors,  department  heads, 


administers,  managers,  board  members,  and  fun- 
ders. The  members  of  the  Advisory  Board  engaged 
in  a twomonth  dialogue  early  on  in  the  program  to  put 
assessment  and  evaluation  objectives  and  proce- 
dures into  place.  Discussions  evolved  around  these 
questions: 

- What  do  we  want  to  evaluate? 

- What  qualitative  and  quantita- 
tive information  do  we  need? 

- What  information  is  already 
available  at  the  nursing  home? 

- How  are  we  going  to  collect, 
analyze,  and  interpret  this  infor- 
mation? 

- Who  will  provide  the  informa- 
tion? 

- Who  will  collect  the  informa- 
tion? 

- When  and  how  often  will  the  in- 
formation be  collected? 

- Which  resources  can  be  allo- 
cated to  the  evaluation? 

- How  will  the  findings  be  used 
and  reported? 

- What  should  the  findings  enable 
us  to  do? 

In  preparation  for  these  meetings,  I listed  the 
program  goals  and  brainstormed  possible  objectives 
and  procedures  to  evaluate  them . Learners  prepared 
as  well.  They  shared  their  assessment  preferences 
as  part  of  the  intake  process.  Participants  filled  out 
questionnaires  about  how  they  liked  to  learn  and 
how  they  liked  to  be  assessed.  The  questionnaire 
was  adapted  from  Nunan2  and  asked  these  assess- 
ment related  questions: 


How  do  you  like  to  find  out  that  your  English  is 
improving?  By 

— written  tests? 

— teacher  evaluation? 

— statements  by  your  supervisor? 

— seeing  if  you  can  use  what  you 
learned  on  the  job  ? 

— checking  your  progress  by  com 
paring  your  work  (eg.  tapes)? 


Do  you  get  a sense  of  satisfaction  from 

- teacher  grades? 

- being  told  that  you  have  made 

progress? 

- keeping  a learning  diary? 

- feeling  more  confident  in  situ- 
ations at  work  and  elsewhere  ? 


Clarifying Assumptions 

As  the  members  of  the  Advisory  Board  shared 
their  evaluation  needs  and  preferences,  the  need 
arose  to  clarify  the  beliefs  about  evaluation  that 
surfaced  in  the  discussion  and  to  reach  a consensus 
on  what  evaluation  should  entail.  These  statements 
reconcile  the  viewpoints  of  those  involved. 

Evaluation  entails  more  than  identifying  the 
individual  and  group  achievements  of  participants. 
A variety  of  procedures  are  needed  rather  than  one 
single  measure  of  outcome.  Depending  on  the  level 
and  purpose  of  evaluation,  activities  are  needed  that 
show  how  learners  and  the  program  are  doing  as  the 
program  is  growing  (ie. , formative  evaluation)  as 
well  as  activities  that  create  opportunities  to  reflect 


on  the  achievements  of  learners  and  the  program  at 
set  times  (ie.,  summative  evaluation).  Evaluation 
activities  provide  qualitative  as  well  as  quantitative 
information  and  both  kinds  of  information  are  consid- 
ered equally  valuable.  Activities  and  procedures  are 
compatible  with  the  assessment  and  evaluation  pref- 
erences of  those  involved.  Evaluation  encompasses 
three  levels:  assessment  of  participants,  evaluation 
of  die  program,  and  evaluation  of  organizational 
change. 

Evaluation  activities  facilitate  the  participation 
of  the  following  people: 

When  assessing  participants 

- participants  themselves 

- teachers 

- nursing  home  residents 

- supervisory  and  licensed  staff 

The  assessment  of  participants  aims  at  identify- 
ing changes  in  learning  and  changes  in  the  transfer  of 
learning  to  the  job. 

Changes  in  these  areas  are  documented: 

- personal 

- affective 

- social 

(within  as  well  as  outside  the  classroom) 

- oral  language  use 

- written  language  use 

- functional  uses  of  English 

in  the  delivery  of  patient  care 

- functional  uses  of  English 
outside  the  nursing  home 

- metacognitive 

-other 

When  evaluating  the  program 

- participants 

- teachers 

- supervisors  and  licensed  staff 


- administrators 

- managers 

- members  of  the  Board  of  Directors 

- funders 

- representatives  from  CACA,  the  learning 
provider 

- members  of  the  Advisory  Committee 

Here  the  focus  of  the  evaluation  is  on  the 
program  itself,  that  is,  all  the  processes  and  resources 
used  to  bring  about  intended  changes  (eg. , program 
design,  the  content  and  method  of  instruction,  mate- 
rials) 

When  evaluating  organizational  changes 

- supervisors  and  licensed  staff 

- program  coordinator 

- administrators 

- representatives  from  management 

This  level  of  evaluation  aims  at  identifying 
organizational  changes  (eg.,  changes  in  the  quality  of 
care)  and  at  clarifying  the  link  between  the  education 
program  and  these  changes. 

Planning 

Inspired  by  Feuerstein’s  participatory  ap- 
proach3 to  evaluating  community  development  pro- 
grams, I then  facilitated  the  design  of  an  evaluation 
plan. 

This  is  the  grid  I used  to  represent  the  different 
planning  decisions  that  needed  to  be  made: 


EVALUATION  PLAN 

Formative: 

What?  Why?  How?  When?  Who?  Conditions? 

Summative: 

What?  Why?  How?  When?  Who?  Conditions? 


To  determine  which  procedures  were  feasible 
or  not,  certain  criteria  simplified  the  decisionmaking 
process.  These  were  key:  the  time  requirement 
needed  to  be  low;  cost  needed  to  be  moderate;  the 
active  involvement  of  workers  and  supervisors 
needed  to  be  assured;  procedures  needed  to  be  able 
to  produce  qualitative  as  well  as  quantitative  infor- 
mation; program  staff  and  learners  needed  to  feel 
comfortable  with  the  procedure;  and  program  staff 
needed  to  have  the  skills  to  implement  them. 

Formative  Evaluation 

These  are  the  formative  evaluation  and  assess- 
ment procedures  and  activities  the  Advisory  Board 
chose  to  find  out  how  learners,  teachers,  and  the 
program  were  doing  along  the  way  and  that  allowed 
for  midstream  monitoring:  feedback  sessions  at  the 
aid  or  during  each  class;  class  observations  by  super- 
visory, licensed,  and  administrative  staff;  peer  obser- 
vation by  teachers;  observation  and  mentor  coach- 
ing by  the  Program  Coordinator  in  weekly  meetings 
with  the  on-site  staff;  meetings  with  the  administrator 
on  a regular  basis  (ie. , biweekly  and  when  the  need 
occurred);  meetings  with  department  heads,  super- 
visors, and  licensed  staff;  progress  notes  by  teachers 
based  on  observation  s within  as  well  as  outside  the 
classroom;  learning  diaries  and  individual  and  group 
journals;  individual  conferencing  with  learners  at  the 
cycles’  beginning  and  end  and  at  several  time 
throughout  the  cycle;  meetings  with  the  Advisory 
Committee  (monthly);  portfolios  of  learners’  work; 
and  learner  writings  and  publications. 

Summative  Evaluation 

These  summative  procedures  and  activities 
were  selected  and  implemented  with  the  completion 
of  each  22-week  cycle:  learner  self-assessments 
using  checklists  of  functional  uses  of  English  as  part 
of  their  individual  education  plan;  teacher  assess- 
ments of  each  learner  using  various  reference  lists  to 
describe  observed  or  reported  changes  and  to  ana- 
lyze the  learner’s  work;  teacher  assessments  of  the 


achievements  of  participants  as  a group  (ie. , per- 
centage of  objectives  achieved  over  cycle);  learner 
evaluations  of  the  program  in  the  form  of  discus- 
sions in  the  learners’  native  language;  supervisory 
and  licensed  staff  assessments  of  learners  (ie.,  trans- 
fer of  learning  to  the  job)  and  evaluations  of  the 
program  in  the  form  of  a questionnaire;  resident 
assessments  of  learners  and  evaluations  of  the  pro- 
gram in  the  form  of  a conversation  with  program 
staff  at  a coffee  and  donut  party;  coordinator  and 
administrator  reviews  of  nursing  home  records  (eg., 
comparison  of  retention  among  participants  and 
non-piarticipants);  and  Advisory  Board  review  of 
the  program  using  key  points  drawn  from  the  activi- 
ties above. 

A Sample:  Learner  Self-Assessment 

Once  a week  in  class  participants  filled  out 
their  learning  diaries.  By  completing  a number  of 
sentences,  learners  documented  on  an  on-going 
basis  what  they  learned  and  studied  and  where  they 
used  English  and  with  whom.  They  also  specified 
what  was  difficult  for  them  and  what  their  learning 
and  practicing  plans  were  for  the  coming  week. 

The  idea  for  the  learning  diaries  came  from 
David  Nunan4.  I adapted  Nunan’s  diary  sheets, 
translated  them  into  Chinese,  made  photocopies, 
and  stapled  the  copies  as  a booklet  with  a cover 
sheet  including  the  participant’s  name  and  cycle. 

Several  times  during  the  cycle  and  at  the 
cycle’s  end,  I asked  participants  to  review  their 
learning  diaries  in  class.  I also  used  the  diaries  in  the 
individual  assessment  of  learners  and  generated 
and  compared  inventories  of  purposes  learners 
used  English  for.  This  allowed  the  learners  and  me 
to  identify  gains  made  over  time. 

These  were  the  statements  I included  in  the 
diaries: 

This  -week  1 studied. . . 

This  week  1 learned. . . 

This  week  1 used  my  English  in  these  places 


This  week  1 spoke  English  with  these  people 
This  week  my  difficulties  were. . . 

I would  like  to  know. . . 

My  plans  for  next  week  are. . . 

Evaluating  Assessment  and  Evaluation 
The  implementation  of  the  initial  evaluation 
procedures  and  activities  revealed  the  benefits  and 
shortcomings  of  the  evaluation  plan  and  necessitated 
its  ongoing  evaluation.  Also,  learners, teachers,  su- 
pervisors, administrators,  and  managers  became 
more  proficient  in  articulating  their  evaluation  needs 
and  preferences  and  priorities  changed  over  time. 

To  accommodate  these  changes,  the  Advisory 
Board  did  not  only  review  the  outcomes  of  the 
evaluation  but  devoted  time  and  energy  to  the  evalu- 
ation of  the  processes  that  generated  the  information 
as  well.  As  the  language  skills  of  learners  developed, 
for  example,  the  learning  diaries  were  replaced  by 
less  structured  dialogue  journals. 

The  on-going  communication  about  assess- 
ment and  evaluation  between  learners,  teachers, 
supervisors,  administrators,  and  managers,  how- 
ever, does  not  mean  that  the  evaluation  plan  cannot 
improve.  On  the  contrary,  the  Advisory  Board  had 
committed  itself  to  continue  to  look  for  ways  to 
assess  and  evaluate  that  specific  to  the  program’s 
goals,  tailored  to  the  resources  of  the  nursing  home, 
conform  with  the  grant  guidelines,  and  are  compat- 
ible with  the  individual  and  cultural  evaluation  needs 
and  preferences  of  those  involved.  This  is,  in  my 
opinion,  where  the  strengths  of  a participatory  ap- 
proach to  evaluation  lies. 
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Getting  in  Touch: 
Participants'  Goals  and  Issues 


by  Lucille  Fandel 


8 """"'""""'IP  rqject  HABLE  (Hispanic  Adults  Basic 
§§  Literacy  in  English)  at  El  Centro  Del 
........  II  Cardenal  provides  basic  English  literacy 

classes  and  basic  mathematics  to  students  with  lim- 
ited English  skills.  The  program  has  four  levels  of 
ESL.  The  experiences  of  levels  one  and  two  are  de- 
scribed in  this  article. 

The  Participants 

Level  one  is  composed  of  people  who  come  to 
the  program  with  little  or  no  English  skills . Partici- 
pants in  level  two  may  have  some  of  the  basics  but 
are  not  yet  able  to  converse  or  write  easily  in  English. 
Both  groups  are  Hispanic  immigrants,  sometimes 
with  little  formal  education,  generally  between  the 
agesof21  and45.  There  are  many  outside  demands 
on  the  learners  as  they  study  English.  Most  are . 
parents.  Some  also  work  outside  the  home.  They 
may  come  to  class  late  if  the  school  bus  fails  to  pick 
up  their  children  or  if  they  have  been  on  an  all  night 
shift.  They  may  not  come  to  class  at  all  if  the  children 
are  home  sick  or  on  school  vacation  or  if  they  are 
asked  to  work  overtime.  They  have  little  time  for 
homework.  Nevertheless,  the  learners  are  highly 
motivated. 

Two  Experiences 

What  follows  are  accounts  of  my  efforts  to  sur- 
face participants’  goals  and  issues  using  two  different 
tools.  I do  this  at  the  beginning  of  each  new  cycle  for 
two  reasons:  First,  as  Elsa  Auerbach  writes,  “the 


essence  of  a participatory  approach  is  centering 
instruction  around  content  that  is  engaging  to  stu- 
dents.” It  is  of  primary  importance  to  uncover 
themes  and  issues  as  an  integral  part  of  classroom 
interaction.  This  co-investigation  “assures  rele- 
vance of  content,”  “shifts  the  balance  of  power” 
toward  students  gaining  more  control  over  their  own 
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learning,  and,  in  itself,  helps  develop  “literate” 
skills.1  A second  important  reason  for  surfacing 
goals  and  issues  is  so  that,  at  die  aid  of  the  cycle  (and 
at  any  point  during  the  cycle)  I can  dialogue  with 
learners  about  their  progress  and  whether  or  not  the 
course  is  meeting  their  needs.  - 

1.  "Mapping  Your  Neighborhood" 

Using  “Mapping  Your  Neighborhood”  a tool 

that  I adapted  from  English  At  Work:  A Tool  Kit  for 
Teachers. , 2 1 ask  participants  to  think  about  where 
they  most  urgently  need  to  be  able  to  use  English 
given  the  various  places  where  they  live,  work, 
recreate,  shop,  go  for  services,  and  visit.  (NOTE: 
With  level  one  students  I use  their  first  language  to 
give  instructions.)  Using  a piece  of  newsprint  and  a 
marker  I then  model  the  drawing  of  pictures  of  some 
of  the  places  I go  being  careful  to  use  examples  the 
learners  would  be  unlikely  to  copy,  (figure  1)1  then 
ask  than  to  work  in  groups  of  three  to  do  a composite 
picture  of  where  they  most  need  English. 

I hang  the  pictures  above  the  blackboard  then 
ask  individuals  to  tell  more  (in  English,  if  they  can) 
about  the  needs  represented  in  the  pictures,  (figure 
3)  I also  record  this  on  newsprint  and  keep  this  list  on 
the  classroom  wall  for  the  duration  of  the  cycle.  Pe- 
riodically, I refer  to  the  list— for  example  as  we  move 
into  a new  theme— to  encourage  the  sense  that  the 
lesson  content  is  relevant  to  the  interests  they  listed 
in  the  beginning  of  the  cycle. 

I also  refer  to  the  newsprint  list  at  the  end  of  the 
cycle  as  part  of  a mutual  evaluation  of  our  accom- 
plishments as  well  as  a reminder  of  skills  yet  to  learn. 

2.  Where  Do  I Want  to  Go? 

Within  the  HABLE  program, each  new  cycle  of 
classes  might  find  some  participants  staying  on  at  the 
same  level  and  some  new  students  joining  them. 
Since  some  students  repeat  the  class  (and  since  I 
enjoy  trying  new  things)  I have  tried  a fresh  approach 
to  surfacing  learners'  issues  and  goals.  This  new  tool 


... in  government  offices 
. . .with  the  receptionist 
... in  an  interview 
. . .in  the  market/store 
. . .how  to  ask  for  what  you  want 
. . . how  to  return  or  exchange  things 
. . . how  to  get  your  money  back 
. . .if  the  police  stop  you 
. . .in  the  hospital 
. * .with  the  receptionist 
. . .with  the  person  doing  the 
intake  interview 
. . . telling  where  the  pain  is 
...  at  home 
. . .with  my  wife 
. . . looking  for  work 
. . . filling  in  an  application  form 
. . . answering  questions  in  an  interview 
...  at  work 

. . .talking  with  the  boss 
. . . asking  for  overtime 
...asking  for  time  off 
...to  report  in  sick  (by  telephone) 

. . .in  an  office 

. . .vocabulary  for  office  items 


Figure  2 

from  the  U.K.  ’s  Center  for  Research  into  Education 
of  Adults.  The  Progress  Profile 3 includes  a “five 
questions,”  a progress  review,  prompt  cards,  and 
tutor  notes. 

The  tool,  according  to  its  authors, 

is  designed  to  help  students,  tutors  and 
the  organization  to  improve  learning 
and  teaching.... It  offers  a framework 
that  will  assist  students  and  tutors  to 
plan,  organize  and  monitor  the  work 
done.. .The purpose.. .is  two  fold.  First, 
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1,  WHERE  DO  I WANT  TO  GO? 


Decide  with  your  tutor  wliat  your  general  goals  are 

X L*.  C60\_  5 fCA't 

To  be  * 


I would  like  to  be  able  to 


beaut 


X W«.vr/  u/' 

5 £^V\^Ls^. 

"ft  V/li-U  Y\6f  Vice*/ 

\Vv\e-V 


I hope  to  adiieve  this  by  1 

Signed . , , ^ ,i)ate_ 


.(date) 


4.  HOW  FAR  HAVE  I GOT? 


^uL  T 

My  Comments 

Write  down  what  you  feel  more  confident  about 
1 CtXA.  u^dt. 'Sri*  fret  <Xyiaf 
R*.aJ  be.  tr&llinn.  Jjifti 
b*t~  to  <A*d-cr$  fa  „!( 

to  5 jj'P&.ft  V-eyk 

$L>XL/!y 


. . . and  what  you  are  stffl  unsure  oL 


.$■  Y\4.<1  /.ec^rh  Ynoye 
b W/t  U a.n*(  5-^e.air 


fohi 

Tutor’s  Comments 
Ask  your  tutor  to  write  down  her  thoughts  on  your  progress. 


n<m  Zf&r&xi 

'OsiSuAth  Sememe 


TObtn&Lj  T-<»*  hafff. 

Merc  uihe*  y*u*  fmod  A*a 
fa  art  **/  Sy**  '•%* 

a MU  harder  y,u  urdt fr*f ress£*Ur 

Tryb  speaKa  &,+*»«* '*  &**• 

X -IhrrtlC  )to  W + Sf&t*'*- 

hfAiff  V*u  {b  3_ • D±_  *'■ 


f0U- 


Agreed  Comments 


'<yhc< 


Discuss  your  progress  with  your  tutor  and  list  the  things  you 
agreed  on. 


Figure  3 
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students’  learning  is  likely  to  be  more 
effective  if  it  is  based  upon  their  own 
definition  of  aims,  and  progress  to- 
wards those  aims.  Second,  [organiza- 
tions] will  be  better  able  to  evaluate 
and  develop  theirprovision  if  progress 
can  be  monitored  within  an  agreed 
framework.”4 

The  view  of  literacy  on  which  the  tool  is  based 
includes  the  ideas  that  there  are  multiple  literacies, 
and  that  coping  in  one  situation  does  not  necessarily 
mean  that  a person  is  “literate”  in  other  situations, 
that  reading,  writing,  listening  and  speaking  are  all 
communication  skills  that  involve  interaction  with 
spoken  and  written  text,  and  that  many  interacting 
factors  affect  our  ability  to  communicate. 

I have  used  and  adapted,  the  “Five  Questions:  ” 

1 . Where  do  I want  to  go? 

2.  What  do  I need  to  learn? 

3.  How  am  I going  to  get  ther? 

4.  How  far  have  I got? 

5.  Where  to  next? 

This  tool  has  served  our  classes  well  in  various 
ways: 

At  the  beginning  of  a cycle... 

- to  let  me  know  from  the  beginning  of  my 
time  with  a particular  group  how  partici- 
pants see  their  language  needs; 

- to  let  participants  know  that  I consider 
their  learning  to  be  their  responsibility-  - 
that  I expect  them  to  set  their  own  indi 
vidual  language  goals;  and 

- to  discover  some  of  their  issues  around 
which  we  can  do  language  work. 

At  the  end  of  a cycle... 

- to  help  participants  look  back  and  assess 
their  own  progress  and 


- to  facilitate  a conversation  between  each 
learner  and  myself  regarding  whether 
they  should  move  on  to  the  next  level  or 
not 

(Note:  If  a standardized  test  was  also  admini- 
stered to  a student  at  the  end  of  a cycle  we  would  also 
talk  about  these  results  too.)  This  would  help  avert  a 
situation  wherein  a student  felt  the  teacher  had 
passed  them  over  for  “promotion”  or  pushed  them 
to  the  next  level  before  they  felt  ready  to  move. 

Figure  3 is  an  example  of  what  was  shared  by  a 
level  two  participant. 

Limitations 

I did  find  some  limitations  to  this  tool: 

- It  is  difficult  for  lower  level  students  to 
understand  what  they  have  to  do  with 
this  tool.  This  may  be  due  not  only  to 
their  more  limited  command  of  the  lan- 
guage but  also,  in  the  case  of  many,  to 
a lack  of  experience  doing  this  kind  of 
planning  exercise.  Higher  level  stu- 
dents have  a much  easier  time  with  it. 

- The  process  was  tedious,  especially  for 
lower  level  students. 

- The  original  material,  as  I received  it,  in 
eluded  some  samples  of  goals  which 
could  be  printed  on  a series  of  different 
index  cards  to  give  ideas  to  those  stu- 
dents who  needed  them.  These  I 
adapted  to  fit  what  I knew  might  more 
likely  be  goals  for  a particular  group, 
(appendix  1) 

- Using  these  “cards”  made  the  exercise 
easier  and  more  satisfying  for  some. 
Others  didn’t  need  them. 

I continue  to  use  this  evaluation  tool  with  inter- 
mediate level  students  as  it  serves  well  to  foster 
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reflection- within  teacher  and  students,  informing 
and  involving  us  all  in  the  learning  process. 
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Read/Write/Now  Adult  Learning  Center 

Assessment  Adventures 


Janet  Kelly 


The  Program 

Read/Write/Now  is  a small,  library-sponsored 
program  for  adult  beginning  readers  and  writers  in 
the  north  end  of  Springfield,  Massachusetts.  It  was 
started  in  1987  as  a pilot  project  with  a grant  from  the 
Massachusetts  Board  of  Library  Commissioners  to 
provide  literacy  services  and  to  test  a packaged  com- 
puter literacy  curriculum.  Since  then,  ithas  become 
a whole  language-based  adult  learning  center  which 
uses  its  six  computers  for  producing  and  publishing 
learners’  writing. 

Curriculum  and  program  development  are  con- 
tinual processes  that  involve  the  contributions  and 
collaboration  of  many  talented  teachers  and  commit- 
ted learners.  The  materials  that  are  included  here  are 
a result  of  this  collaboration  between  teachers  and 
learners.  There  is  a much  bigger  file  of  things  we  no 
longer  use  than  of  things  we  use.  If  it  does  not  work, 
we  ask  ourselves  if  we  need  it  and  if  there  is  a good 
answer,  we  revise  it.  If  it  still  does  not  work,  we  put 
it  in  the  archives.  I consider  these  assessment  mate- 
rials works  in  progress  in  the  sense  that,  although 
they  have  been  tried  and  revised,  there  is  always 
room  for  improvement. 

We  have  tried  to  develop  assessment  tools  that 
work  for  learners  and  teachers  as  well  as  funding 


sources.  What  follows  are  some  samples  of  assess- 
ment tools  and  descriptions  of  how  we  have  used 
them  for  initial  and  early  assessment.  Materials  for 
ongoing  assessment  and  "then  Sc  now"  assessment 
are  referred  to  but  not  included  with  this  article. 1 

Assessment  in  a Whole  Language 
Classroom 

Assessment  in  a whole  language-based  class- 
room is  ongoing.  It  is  an  integral  part  of  the  program 
design  and  helps  drive  the  process  of  developing 
curriculum.  One  of  the  goals  of  a whole  language 
program  is  to  empower  learners  by  helping  them  to 
become  more  self-directed,  to  identify  and  work 
towards  their  own  goals  for  learning,  literacy,  and 
life.  A logical  outgrowth  of  that  goal  is  to  incorporate 
learners  ’ self-evaluation  into  the  overall  assessment 
activities  of  a program.  Both  learners  and  teachers 
need  to  know  why  they  do  what  they  do  in  a 
classroom  so  that  they  can  have  a sense  of  progress, 
as  well  as  make  decisions  about  future  directions. 
One  great  motivator,  especially  in  something  as 
challenging  as  adult  basic  education,  is  for  learners 
to  clearly  see  and  feel  their  own  growth.  Standard- 
ized tests  do  not  measure  growth  in  self-esteem , life 
skills,  empowerment,  community  involvement,  and 
self-confidence,  nor  do  they  truly  measure  the  liter- 
acy skills  that  adult  learners  make  in  many  programs. 

Evaluation  in  a whole  language-based  program 


Thanks  go  to  Doreen  Walsh  and  Don  Robishaw,  two  creative  teachers  who  worked  at  Read/Write/Now  throughout  this  project. 
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is  a learning  activity  as  well  as  a process  of  adding  to 
information  about  learners’  progress,  teachers’  ef- 
fectiveness, and  ideas  for  curriculum  development. 
Whole  language  assessment  includes  observation, 
interviews,  learners’  self-evaluations,  and  formal 
and  informal  analysis.  Examples  of  each  of  these 
aspects  of  evaluation  are  included  in  this  article. 

Alternative  assessment  tools  which  are  learner- 
centered  and  holistic  rather  than  standardized  and 
quantitative  are  being  developed  and  used  in  many 
programs.  Some  of  these  programs  describe  them- 
selves as  whole  language,  some  have  a Freirean  ap- 
proach, and  others  have  an  eclectic  approach  to 
literacy  education.  All  are  concerned  with  evalu- 
ation for  the  purposes  of  increasing  learners’  and 
teachers’  sense  of  empowerment  as  participants  in 
the  process  of  learning  and  explicitly  understanding 
their  own  growth  and  progress  towards  meeting  their 
goals.  This  kind  of  assessment  is  not  concerned  with 
measuring  changes  in  behavior,  but  may  use 
changes  in  behavior  as  evidence  of  increasing  un- 
derstanding, knowledge,  and  competence. 

The  question  we  try  to  keep  asking  ourselves 
about  assessment  activities  is  WHY?  There  should 
always  be  an  answer  to  that  question  that  relates  to 
helping  learners  and  teachers  become  empowered 
and  active  participants  in  the  process  of  their  own 
education  and  work.  If  there  is  no  good  answer  to  the 
WHY  question,  or  the  answer  does  not  relate  to  the 
development  of  the  program,  the  benefit  of  the 
learner  and  the  teacher  as  members  of  a learning 
community,  it  is  time  to  scrap  the  particular  assess- 
ment activity. 

Portfolio  Assessment 

Read/Write/Now  uses  a portfolio  approach  to 
maintain  a comprehensive  record  of  ongoing  assess- 
ment. A learner’s  assessment  portfolio  contains: 
initial  screening  and  placement  interview,  including 
a first  writing  sample,  middle  and  later  pieces  from  a 
learner’s  writing  folder,  dialoguejoumals,  learning 


contracts,  goals  checklist,  writing  conference  rec- 
ords, writing  progress  checklists,  reading  miscue 
analysis,  book  lists,  reading  conference  records, 
student  learning  logs  and  other  self-assessment 
materials,  and  publications  of  the  learner’s  writing. 
Anecdotes  and  observations  related  to  learners’ 
progress  are  often  kept  in  a running  teacher’s  log  and 
are  included  as  part  of  the  portfolio,  (figure  1) 

The  learner’s  portfolio  is  added  to  and  shared 
with  the  learner  during  regular  conferences 
throughout  the  cycle  and  at  other  times  when  a 
learner  needs  to  see  concrete  evidence  of  his/her 
progress.  The  portfolio  can  also  be  used  to  document 
progress  to  funders.  Although  this  approach  tends  to 
be  more  comprehensive  than  many  funders  have 
time  or  interest  in  seeing,  it  helps  to  organize  evalu- 
ation materials  in  away  that  makes  writing  narrative 
summaries  for  funding  reports  easier. 

Sample  materials  and  notes  fordoing  initial  and 
early  assessment  follow.  (See  appendix  2 for  a 
complete  inventory  of  assessment  tools  used  at 
Read/Write/Now.) 


Initial  Assessment 
(before  learner  enters  program): 
Initial  Screening  & Placement  Interview, 
Slosson  Oral  Reading  Test 
Teacher-made  comprehension  exercise 

Early  Assessment 

(first  3 weeks  of  class  cycle): 

Goals  Checklist 

Modified  Burke  Reading  Interview 
Looking  at  Your  Own  Reading 
Behavior 
Miscue  Analysis 
Reading  Progress  Checklist 
Writing  Progress  Checklist 
Learning  Contract 
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WHAT'S  IN  A PORTFOLIO? 


GOAL  SETTING: 

initial  screening  & placement  interview, 
including  Slosson  score  & responses  to 
teacher-made  comprehension  exercise 
goals  checklist 
learning  contract 


READING: 
V"  list  of  books  read 
reading  progress  checklist 
looking  at  your  own  reading  behavior 
modified  Burke  interview 
reading  conference  record 
miscue  analysis  results 


WRITING: 

dialogue  journal  samples 
first  writing  sample 
other  writing  samples,  dated 
writing  conference  record 
writing  progress  checklist 
published  writings 


OTHER: 
learning  logs  (student’s) 
teacher’s  logs  (anecdotes/observations) 
program  and  learner’s  self-evaluation  materials 
Slosson  post-test,  if  applicable 
math  activities  record 
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Screening  Process 

Initial  Contact.  There  is  generally  a 10-20 
minute  telephone  contact,  before  potential  new  stu- 
dents come  in  for  a screening  interview,  during 
which  learners  have  a chance  to  tell  their  story  and 
describe  their  needs  while  a staff  person  listens  and 
offers  encouragement.  If  it  seems  appropriate,  the 
staff  person  describes  the  program  and  the  intake 
process.  If  the  learner’ s needs  cannot  be  met  by  our 
program,  care  is  taken  to  make  referrals  to  other  re- 
sources and  the  learner  is  encouraged  to  call  back  for 
more  help  if  needed.  If  appropriate,  a screening  in- 
terview is  scheduled  at  a time  that  is  convenient  for 
learner  and  staff  and  bus  route  information  and  direc- 
tions are  given. 

Screening/Placement  Interview.  This  usu- 
ally takes  30-45  minutes,  sometimes  longer.  The 
overall  purpose  of  this  interview  is  to  make  learners 
feel  positive  about  their  decision  to  pursue  their 
goals  for  literacy,  to  introduce  them  to  the  way  the 
program  works,  and  to  get  a picture  of  the  learners' 
goals,  experience  with  education,  and  some  of  the 
skills  they  will  bring  to  learning. 

The  interview  is  conducted  in  this  framework: 

a.  Make  the  learner  feel  as  comfortable  as  pos- 
sible; this  may  involve  offering  coffee,  acknowledg- 
ing the  fact  that  s/he  has  already  done  the  hardest 
thing,  which  was  to  take  the  first  step  by  coming  to 
the  interview. 

b.  Tell  the  learner  what  will  happen  and  how 
long  it  will  take.  Encourage  the  learner  to  ask  ques- 
tions at  any  point.  Assure  her/him  that  everything 
s/he  discusses  with  the  interviewer  is  confidential 
and  that  if  s/he  is  uncomfortable  at  any  time  in  an- 
swering a question  or  doing  something  that  is  part  of 
the  interview,  to  say  so. 

c.  Ask  the  learner  if  s/he  would  mind  writing 
her/his  name,  address,  and  phone  number  on  a sheet 
of  paper.  Offer  help  as  needed.  This  serves  the  dual 


purpose  of  getting  information  and  letting  a learner 
who  can  write  these  things  without  help  feel  good 
about  doing  it  If  a learner  cannot  write  her/his  name, 
it  lets  the  interviewer  know  how  to  adapt  some  of  the 
other  tasks  involved  in  the  rest  of  the  interview. 

d.  A three  page  interview  form  (figure  2,  appen- 
dix 3)  is  used  to  ask  the  learner  about  her/his  goals, 
uses  for  reading  and  writing  in  her/his  life,  and  expe- 
riences with  learning  in  the  past.  The  interview  also 
includes  some  questions  designed  to  get  at  the 
learner's  commitment  to  learning  and  any  barriers 
s/he  feels  s/he  will  need  to  overcome  to  reach  her/ 
his  goals.  The  last  item  on  the  interview  is  a program 
policy  statement  regarding  use  of  drugs  or  alcohol 
during  class  time  by  staff  or  learners  which  requires 
the  signatures  of  learner  and  interviewer.  The  inter- 
view form  is  used  as  a guide,  with  staff  asking  the 
questions,  adapting  them  as  needed,  and  writing  the 
learner’s  responses  on  the  form. 


read/wrttr/now  screening  a placement  interview 

■'  Otic 

tntovkwu. 


I.  Wh at  b year  most  Important  reason  tor  warning  to  leans  to  read  and  writ*  better? 


2.  W hat  trt  nm  things  that  yon  want  to  do  that  beta*  abte  to  read  and  writ*  better  will 
help  700  to  do? 


3-  yon  hav*  any  children  or  grandchildren  in  yoor  lift?  If  so,  are  they  school  age? 
What  are  their  ages  and  grades  la  school? 


< Do  yon  ha«*  a Job  otw?  If  so,  what  Usd  of  work  do  yon  do?  I r not.  what  kinds  of  jobs 
haea  Too  had?  Do  yon  need  to  os*  reading  and  writing  at  work? 
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e.  Describe  the  approach  that  the  program  takes, 
what  classes  are  like,  and  encourage  the  learner  to 
ask  any  questions  s/he  has  at  this  point. 

f.  Teacher-made  reading  assessment  ex- 
ercise2. For  learners  who  were  able  to  write  their 
name  and  reported  some  reading  ability  and  experi- 
ence in  die  interview,  this  is  the  next  step.  A teacher- 
made  reading  exercise  is  used  to  get  a very  general 
picture  of  a learner’s  comprehension,  fluency  and 
word  recognition  skills.  This  can  be  done  in  a 
number  of  ways.  One  way  is  to  use  a reading  sample 
that  is  accessible  to  most  adult  learners  (1.5  - 2.5 
grade  level),  which  is  written  in  a natural,  predictable 
style,  and  is  a connected,  comprehensible  text.  We 
have  used  a selection  from  the  Tana  Reiff  book,  The 
Door  is  Open , (Fearon) . (figure  3 and  4)  (See  appen- 
dix 4 for  Silent  or  Oral  Reading  Comprehension 
Quick  Assessment  Exercise.)  Because  it  is  about  a 


/ 

I 


woman  returning  to  school  after  many  years,  learn- 
ers can  often  identify  with  her  fear  in  meeting  the 
challenge.  Discussing  the  reading  becomes  an  op- 
portunity to  discuss  the  learners'  own  feelings  about 
coming  to  school  as  an  adult.  It  is  often  the  first  book 
learners  ask  to  take  home  with  them.  It  is  written  at 
a 1.6  - 2.9  grade  level,  according  to  the  publisher,  and 
we  have  found  it  is  accessible  to  most,  though  not  all 
of  the  learners  we  meet. 

Be  sure  to  let  the  reader  know  what  this  testing 
is  for  and  how  it  will  work.  Explain  that  the  reading 
sample  will  be  hard  for  some  people,  and  easy  for 
others,  and  that  it  is  only  a small  part  of  assessing  their 
reading  skills.  Introduce  the  sample  to  the  learner  by 
showing  her/him  the  book,  telling  her/him  the  title, 
looking  at  the  cover  together,  and  previewing  the 
piece  s/he  will  be  reading.  (See  appendix  5 "Some- 
thing We're  Working  On"  for  using  learner-gener- 
ated materials  for  reading  assessment.) 

g.  Siosson  Oral  Reading  Test.  For  learners 
who  were  able  to  read  the  sample  for  the  compre- 
hension exercise  with  some  fluency,  the  next  step  is 
the  Siosson  Oral  Reading  Test  (figure  5,  appendix  6) 
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The  interviewer  explains  the  test;  that  it  isn * t always 
easy  to  read  isolated  words  on  a list,  the  test  isn’t 
perfect,  but  it’s  another  part  of  the  picture  of  what 
skills  someone  has  for  reading.  The  word  lists  are 
given  to  the  learner  one  at  a time  and  s/he  is  asked  to 
read  the  words  s/he  knows  out  loud.  The  interviewer 
marks  each  word  on  a separate  sheet,  using  a plus  (+) 
for  correct  words  and  minus  (-)  for  incorrect  words. 
When  a learner  mates  seven  miscues  on  any  one  list, 
the  testing  stops.  (Complete  scoring  instructions  are 
on  the  test  forms,  appendix  6.)  After  the  stopping 
point,  the  interviewer  often  comments  on  the  skills 
the  reader  showed,  such  as  knowing  the  beginning 
consonant  sounds. 

The  results,  for  the  purpose  of  screening  and 
placing  learners  in  our  program,  are  useful  but  lim- 
ited. The  Slosson  yields  a grade  level  equivalent, 
which  has  been  important  for  some  funding  sources. 
It  also  gives  a limited  picture  of  learners’  decoding, 
pronunciation,  and  sight  word  recognition  skills.  The 
greatest  value  of  using  this  test  for  us  is  that  the 
“grade  level”  number  is  often  an  accurate  gauge  of 
the  level  of  material  a learner  will  be  able  to  read  in- 
dependently. We  find  that  learners  who  start  below 
the  “2.0”  level  as  reflected  on  the  Slosson  pre-test 
and  go  on  to  make  a great  deal  of  progress  in 


connected  reading,  add  to  their  sight  word  vocabu- 
lary and  word  recognition  skills,  and  begin  to  see 
themselves  as  readers  and  writers,  do  not  necessar- 
ily have  that  progress  reflected  in  a Slosson  post-test. 
The  Slosson  is  best  used  as  a placement  test  and  not 
to  measure  significant  growth  and  progress.  How- 
ever, when  we  have  used  it  as  a post-test  for  funding 
sources,  we  have  included  other  documentation  of 
reading  development  for  all  learners,  but  especially 
for  the  lower  level  readers  whose  growth  hardly 
shows  on  the  test. 

h.  Letter  Identification:  For  a learner  who  is 
not  able  to  read  any  of  the  words  on  the  first  list  of  the 


letter  chart 

A8C0EFQhi  JKLMN 
OPQRST  u V w x y z 


Slosson,  or  who  has  indicated  s/he  cannot  read  or 
write  at  all,  the  interviewer  may  ask  if  s/he  knows 
some  of  the  letters  of  the  alphabet.  A letter  chart  is 
used  to  have  the  learner  point  out  the  ones  s/he 
knows.  Some  learners  will  report  seeing  things 
backwards  or  having  been  told  they  are  dyslexic.  A 
letter  chart  might  be  used  to  find  out  which  letters,  if 
any,  learners  reverse.  If  this  is  a severe  problem, 
consultation  is  sought  with  a learning  disabilities  spe- 
cialist for  further  assessment. 

i.  Writing  Sample:  The  last  part  of  the  Screen- 
ing/Placement process  is  to  ask  the  potential  student 
to  try  to  write  something.  This  is  often  not  the 
learner’s  favorite  part.  The  interview  discusses  the 
fact  that  writing  is  hard  and  that  spelling  is  a part  of 
writing  that  everyone  finds  difficult.  The  learner  is 
encouraged  to  try  it,  and  not  to  worry  about  their 
handwriting  or  spelling.  A suggestion  is  made  that 
they  might  want  to  write  about  their  goals  for  them- 
selves in  connection  with  reading  and  writing,  be- 
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cause  it’s  probably  on  their  minds;  but  anything  they 
choose  to  write  about  is  fine.  They  are  seated  at  a 
table,  given  a lined  sheet  and  a pencil  with  an  eraser, 
and  encouraged  to  write  as  much  or  as  little  as  they 
want,  and  to  do  the  best  they  can.  (figure  6,  appendix 
7)  The  interviewer  usually  walks  away  to  do  some- 
thing else  while  the  learner  writes  for  as  long  as  s/he 
wants.  Most  learners  who  can,  do  write.  Those  who 
have  not  been  able  to  write  their  names  are  not  asked 
to  do  this  by  themselves.  They  may  be  asked  if  they 
would  like  to  dictate  something  about  their  goals  and 
then  have  it  read  back  to  them  as  a sample  of  the  way 
they  will  be  learning  to  read  in  class . Learners  who 
write  something  independently  are  encouraged  to 
read  it  to  the  interviewer  and  are  given  positive 
feedback  on  their  courage  and  skills.  Often  the 
interviewer  focuses  on  the  content  of  the  writing  in 
making  positive  comments.  The  writing  sample 


helps  to  assess  what  skills  a learner  will  bring  to 
writing  as  well  as  give  us  an  idea  of  a learner’s  under- 
standing of  the  writing  process . It’ s often  an  oppor- 
tunity to  surprise  and  please  the  writer  by  being  able 
to  read  what  they  write  and  giving  them  support  for 
writing  it  It  is  also  helpful  in  documenting  a learner’s 
progress.  In  part,  we  like  to  have  a writing  exercise 
in  the  intitial  screening  as  an  introduction  to  what  will 
be  happening  in  class  each  day:  writing  will  be  part 
of  it 

j.  Closure:  If  a decision  can  be  made  on  the 
spot,  it  is.  If  not,  a clear  message  is  given  to  the 
learner  as  to  when  a decision  will  be  made.  Learners 
who  are  appropriate  for  the  program  are  invited  to 
join  a class  and  if  they  accept,  a card  with  the  time  and 
date  of  their  first  class  and  the  telephone  number  of 
the  program  is  given  to  them.  Transportation  and 
other  concerns  are  dealt  with  at  this  time.  Learners 
are  told  they  need  to  call  if  they  are  unable  to  come 
to  class.  If  they  wish  to  borrow  a book  before  class 
starts,  they  can  sign  one  out.  Learners  who  are  not 
appropriate  for  the  program  are  told  why  and  given 
a list  of  resources  that  might  better  meet  their  needs . 
Often  these  learners  are  identified  before  the  end  of 
the  interview  and  not  made  to  complete  it. 


EARLY  - ASSESSMENT 


Read/Write/Now  Goals  List3 

Using  a goals  checklist  enables  us  to  help  learn- 
ers identify  their  interests  and  name  their  particular 
goals,  a crucial  step  towards  self-directed  learning, 
(figure  7,  appendix  8)  Learner’s  responses  to  the 
checklist  are  used  to  articulate  goals  and  the  steps  to 
reach  them.  The  goals,  in  turn,  are  put  translated  into 
action  plans  through  short-term  learning  contracts.  It 
is  also  a tool  for  curriculum  development,  since  our 
curriculum  is  based  in  part  on  the  identified  needs 
and  interests  of  participants  in  the  program.  The 
common  interests  that  are  identified  by  the  goals 
lists,  such  as  driver’s  education,  voting  information, 
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or  using  a checking  account,  can  often  be  developed 
into  an  extended  small  group  learning  activity. 

We  have  used  the  goals  list  in  a variety  of  ways. 
It  is  always  part  of  the  activities  early  in  the  class 
cycle.  We  usually  have  a group  lesson/discussion  on 
goal  setting.4  We  have  also  used  readings  from 
learner-written  publications  or  oral  histories  which 
touch  on  educational  experiences  and  life  experi- 
ences that  many  learners  can  identify  with  as  a 
starting  place  for  analysis  and  discussion  about 
where  we  have  been,  where  we  want  to  go,  and  how 
we  can  get  there. 

We  have  introduced  the  goals  list  to  the  whole 
group,  and  also  to  small  groups  facilitated  by  teach- 
ers and  tutors.  The  group  goes  through  each  section 
of  the  checklist  as  learners  check  off  their  own 


responses.  The  teacher  is  available  to  help  as 
needed.  In  some  cases  the  facilitator  also  reads  and 
writes  the  learner’s  responses  on  the  form.  We 
hesitated  about  doing  the  checklists  in  groups  be- 
cause it  seemed  like  a potentially  revealing  situation 
that  might  make  some  learners  uncomfortable. 
However,  it  turned  out  to  be  another  opportunity  for 
learners  to  offer  each  other  support  and  comfort  as 
they  shared  some  of  their  problems  and  hopes 
around  reading  and  writing.  Sharing  their  responses 
was  a spontaneous  occurrence,  not  orchestrated  by 
teachers,  and  those  few  learners  who  felt  uncomfort- 
able in  doing  so  were  able  to  just  write  their  answers, 
ask  for  assistance  from  the  teacher  if  needed,  and 
hand  over  the  finished  checklist.  Obviously,  goals 
checklists  can  also  be  done  on  an  individual  basis, 
with  a teacher  or  tutor  assisting  as  needed  while  one 
learner  goes  through  the  items  and  marks  her/his  re- 
sponses. 

Burke  Reading  Interview  - Modified5 

What?  An  interview  that  explores  the  learner’s 
understanding  of  what  reading  is  and  what  a good 
reader  does. 

Why?  To  help  teacher  and  learner  understand  what 
ideas  about  reading  a learner  is  starting  with;  to  aid  in 
planning  reading  activities  and  discussions  designed 
to  explore  the  reading  process  and  metacognative 
aspects  of  reading  development. 

How?  Usually  done  individually,  with  teacher  act- 
ing as  interviewer  and  scribe;  can  be  adapted  to  a 
group  activity. 

When?  Early  in  class  cycle;  can  be  repeated  at  end 
of  cycle-comparison  of  responses  can  be  interesting 
for  learner  to  reflect  on.  (appendix  9) 

Looking  at  Your  Own 
Reading  Behavior 

What?  Self-assessment  questionnaire  for  learners 
Why?  To  help  learners  become  more  aware  of  their 
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own  reading  behavior  as  part  of  the  process  of  setting 
own  goals  for  reading  and  becoming  more  self- 
directed  learners 

How?  Individually  answered  after  group  discus- 
sion & clarification  of  questions 

When?  After  2-3  weeks  of  classes  in  a cycle;  can 
be  repeated  at  end  of  cycle  for  purposes  of  compar- 
ing responses  to  assess  growth,  (appendix  10) 


Reading  Progress  Checklist 


What?  Teacher’s  short  checklist  for  assessment  of 
individual  learner’s  reading  behavior  and  develop- 
ment over  period  of  class  cycle 


Why?  To  organize  and  categorize  reading  assess- 
ment information  in  an  accessible  format  to  help 
teachers  plan  strategy  lessons  and  activities  for  indi- 
viduals and  reading  groups 


How?  Teacher  considers  learner’s  reading  behav- 
ior in  class,  reading  conferences,  miscue  analysis, 
learner’s  self-assessment  and  responses  to  modified 
Burke  interview  and  completes  checklist 

When?  After  the  learner  has  been  in  classes  for 
about  three  week  and  after  above  assessments  are 
done;  repeat  at  end  of  the  cycle  to  assess  changes, 
(appendix  11) 


Writing  Progress  Checklist6 

What?  Checklist  form  to  assess  various  aspects  of 
writing  development,  both  in  terms  of  writing  skills 
and  self-concept  as  a writer 

Why?  To  provide  a coherent,  comprehensive  pic- 
ture of  a writer’s  progress  for  the  learner  and  for 
teachers  to  assess  individual  and  group  needs  for 
mini-lessons 

How?  Can  be  used  by  teacher  after  looking  at  a 
writer’s  dialogue  journal  and  writing  folder  and 
shared  with  learner  afterwards,  or  filled  out  with  the 


learner,  looking  over  the  learner’s  writing  together 

When?  After  a few  weeks  in  classes,  or  when  the 
learner  has  been  writing  for  three  to  four  weeks; 
repeat  and  compare  results  at  end  the  of  the  cycle, 
(appendix  12) 


Miscue  Analysis 

At  the  core  of  whole  language  reading  assess- 
ment is  the  Reading  Miscue  Inventory,  (appendix 
13)  developed  by  Goodman,  Burke,  and  Watson  and 
adapted  by  practitioners  in  many  settings.  Miscue 
analysis  looks  at  the  effectiveness  of  a reader’s  use 
of  strategies  and  cues  in  making  sense  of  text. 
Inefficient  use  of  language  cues  and  strategies  inter- 
feres with  reading  comprehension,  which  is  the 
heart  of  the  reading  process . The  purpose  of  miscue 
analysis  is  to  discover  the  reader's  strengths  in  using 
various  reading  strategies  to  make  sense  of  texts  and 
to  assess  which  strategies  the  reader  needs  to  de- 
velop and  strengthen  to  improve  his/her  fluency  and 
reading  comprehension.  The  results  of  miscue 
analysis  can  be  used  by  teachers  to  plan  strategy 
lessons  and  reading  activities  which  address  the 
particular  needs  of  readers.  Results  can  also  be 
reviewed  with  students  to  help  them  understand 
their  own  miscues  strategies.  Generally,  miscue 
analysis  is  done  at  times  when  it  is  useful  to  assess  the 
strengths  and  strategies  a learner  has  and  address  the 
areas  that  are  not  yet  developed.  We  try  to  do  this 
early  in  a cycle  of  classes  so  that  the  information  we 
get  from  it  can  be  put  to  use  in  planning  reading 
strategy  lessons  for  learners.  Subsequent  miscue 
analyses  can  be  used  to  evaluate  growth  in  develop- 
ing certain  reading  strategies  by  comparing  them  to 
earlier  miscues  and  making  a learning  plan  based  on 
that  information.  Miscue  analysis  is  only  effective 
with  readers  who  can  independently  read  con- 
nected texts  that  are  approximately  500  words  long. 
The  text  can  be  written  at  a basic  level,  but  there 
needs  to  be  enough  text  for  the  reader  to  read  for 
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about  15  minutes  and  potentially  make  25  or  more 
miscues.  (See  next  article,  "Down  and  Dirty"  Mis- 
cue  Analysis  for  procedures. ) 

Learning  Contracts: 

Encouraging  Ownership 
of  the  Learning  Process 

At  Read/Write/Now,  we  have  been  experi- 
menting with  using  learning  contracts  for  several 
years,  (figure  9,  appendix  14)  One  of  the  great 
benefits  we  have  found  in  using  them  with  adult 
learners  is  that  they  help  learners  to  feel  more 
directed  and  focused  on  their  goals,  and  therefore  to 
have  more  success  in  meeting  them.  The  contract 
can  organize  and  articulate  a learning  plan  that  may 
be  happening  anyway,  but  it  is  clarified  and  sup- 
ported by  writing  it  down.  Learning  contracts  help 
teachers,  too,  in  the  sense  that  they  help  us  to  be 
aware  of  individual  interests  and  needs  of  learners  in 
the  process  of  negotiating  and  monitoring  the  con- 
tracts. Another  positive  effect  of  learning  contracts 
is  an  increased  sense  of  ownership  that  many  learn- 
ers develop  about  the  process  of  their  own  educa- 
tion. It  is  impossible  to  see  education  as  the  passive 
receiving  of  information  from  others  when,  as  a 
learner,  you  have  written  down  a plan  of  action  for 
meeting  your  own  goals  in  reading  and  writing  and 
the  plan  involves  you  doing  things,  notjust  listening 
while  a teacher  tells  you  about  doing  things. 

Learners  are  encouraged  to  name  goals  for 
themselves  to  work  on  in  class  and  outside  of  class. 
We  usually  do  a whole  group  brainstorm  to  come  up 
with  a list  of  what  kinds  of  reading,  writing,  or  math 
activities  people  can  do  when  they’re  not  in  class. 
The  most  creative  ideas  have  often  come  from 
learners  who  are  already  doing  a lot  of  reading  or 
writing  on  their  own,  at  home,  at  work,  anywhere 
and  everywhere.  Quite  a few  of  our  students  have 
taken  copies  of  their  learning  contracts  home  with 
them  to  put  them  up  on  the  refrigerator  to  remind 
themselves  of  what  they  decided  to  do.  The  learning 


contract  is  an  agreement  between  a learner  and  a 
teacher,  and  also  between  a learner  and  her/himself. 
The  learner  and  teacher  together  decide  on  some 
personal  goals  for  reading,  writing,  and  other  goals. 
Two  questions  are  answered  about  each  goal:  What 
do  you  want  to  do?  How  do  you  want  to  do  it? 

The  learning  contract  period  is  usually  about 
eight  weeks.  A short  Learning  Contract  Review 
conference  is  set  up  at  the  end  of  the  eight  week 
period  to  discuss  how  the  learner  and  teacher  think 
the  contract  went  - were  the  goals  met?  Why  or  why 
not?  Are  there  some  better  ways  to  try  to  meet  the 
goals  that  weren’t  meant?  Does  the  learner  want  to 
continue  with  the  same  goals  or  change  some  or  all 
of  them?  At  that  point  a new  contract  is  written  up  or 
notations  are  made  on  the  standing  one  to  indicate 
that  it  is  still  in  effect.  Our  experience  has  been  that, 
for  most  learners,  each  successive  contract  period 
seems  to  make  it  easier  for  learners  and  teachers  to 
fine-tune  the  process  of  naming  reachable  goals  and 
reasonable  steps  to  meeting  them.  The  first  contract 
conference  is  always  the  hardest  in  terms  of  making 
it  a process  that  is  really  owned  by  the  learner. 
Learners’  ownership  of  their  own  learning  seems  to 
develop  over  time  with  the  experience  of  setting  and 
meeting  goals:  The  more  ownership  learners  feel 
over  their  own  education,  the  more  success  they 
seem  to  have  in  meeting  learning  goals,  and  the  more 
success  they  have  in  meeting  goals,  the  more  own- 
ership they  feel  for  their  own  learning. 

Goal  Setting  and  Learning  Contracts 

The  steps  outlined  below  typically  take  place 
after  a learner  has  been  in  the  program  for  three  to 
four  weeks  and  has  had  time  to  get  comfortable  in 
the  group,  get  used  to  the  way  the  program  works, 
and  identify  some  goals,  likes  and  dislikes.  Subse- 
quent Learning  Contract  Review  Conferences  do 
not  require  quite  as  much  preparation  time  on  the 
part  of  the  teacher. 
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A.  Before  Meeting  with  the  Learner: 

1.  Review  learner’s  Initial  Screening  & Place- 
ment Interview,  including  first  writing  sample  and 
answers  to  interview  questions  relating  to  goals. 

2.  Review  learner’s  Goals  Checklist  for  inter- 
ests, specific  goals  named  by  student. 

3.  Writing  Assessment:  Review  dialogue  jour- 
nal and  writing  folder,  using  Writing  Progress 
Checklist  to  help  categorize  progress;  add  categories 
as  needed. 

4.  Reading  Assessment:  Teacher’s  observa- 
tions of  learner’s  reading  behavior  and  develop- 
ment, together  with  these  three  tools: 


a.  Miscue  Analysis:  do  one  or  review  the 
latest  one  done  to  see  where  learner’s  strengths  are 
and  which  strategies  need  to  be  developed  further. 


b.  Looking  at  Your  Own  Reading  Behav- 
ior: look  over  learner’s  responses  and  write  a few 
notes  to  yourself  as  to  any  reading  goals  students  may 
have  stated  or  implied  in  their  answers. 

c.  Burke  Reading  Interview  (Modified): 
for  teacher’s  use  in  understanding  how  a learner 
views  the  reading  process,  which  may  help  in  plan- 
ning activities  for  that  person  or  the  reading  group. 

d.  Reading  Progress  Checklist:  for 
teacher’s  use  in  assessing  individual  reader’s  prog- 
ress as  observed  in  reading  groups  and  individual 
conferences. 

B.  Orientation  to  learning  contracts  is  done 
as  a group  activity  before  conferences  begin. 

C.  Individual  Conferences: 

1.  Materials  needed:  Learner’s  assessment 
portfolio,  learning  contract  form. 

2.  Procedure: 

a.  Review  purpose  of  conference  & con- 
tract, briefly. 

b.  Explain  timeframe  (about  eight 
weeks)  for  contract  and  need  to  choose  short-term 
goals  that  can  be  accomplished  in  that  time. 

c.  Learners  may  have  several  reading  and 
writing  goals,  some  for  in  class  and  some  for  outside 
of  class. 

D.  Choosing  Goals: 

Learning  about  helping  learners  to  choose  goals 
and  trying  to  empower  them  as  decision-makers  in 
their  own  learning  has  been  an  evolutionary  process 
since  we  first  began  using  goals  Checklists  and 
learning  contracts  several  years  ago.  We  have  gone 
from  a stance  of  very  limited  interference  and  influ- 
ence in  the  process  of  learners’  decision-making 
about  their  educational  goals  to  the  role  of  full 
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participants  in  a learning  community.  Full  participa- 
tion means  listening  to  each  other,  sharing  our  opin- 
ions, knowledge,  and  advice  in  the  process  of  nego- 
tiating the  decisions  that  we  often  make  together. 
Setting  goals  as  individual  learners  and  deciding 
what  our  goals  as  a group  are  going  to  be  are 
negotiated  decisions.  Teachers  and  learners  both 
have  a voice  in  the  way  it  all  turns  out.  If  we  did  not 
negotiate  over  these  decisions,  there  would  be  a 
greater  rate  of  frustration  among  both  learners  and 
teachers.  If,  for  instance,  a learner  who  could  only 
read  a few  words  decided  he  was  going  to  get  his 
driver’s  license  within  the  contract  period  of  8 weeks 
and  the  teacher  meeting  with  him  simply  said  that 
sounded  like  a fine  goal,  without  considering  what 
kinds  of  materials  were  available  and  how  much  time 
it  would  take  the  learner  to  have  enough  understand- 
ing of  the  material  covered  in  the  test  to  take  the  oral 
exam,  chances  are  the  learner  would  not  meet  his 
goal  and  the  teacher  would  not  feel  like  she  had  done 
her  best  to  help  him  meet  it.  What  we  have  learned 
about  goal  setting  is  that  both  the  learner  and  the 
teacher  need  to  have  input  into  the  process.  Goals 
are  made  to  be  reached,  so  the  more  realistic  we  can 
be,  the  better  all  concerned  feel  about  the  process. 
But  it  is  a delicate  balance  - longer  term  goals  and 
aspirations  need  to  be  nourished.  All  of  us  need  far 
off  goals  to  aim  for,  and  all  great  accomplishments 
start  one  step  at  a time.  As  teachers,  we  try  to  help 
learners  figure  out  what  steps  they  can  start  now  to 
get  to  where  they  want  to  go,  whether  they  arrive 
next  week  or  next  year. 

1.  Start  with  what  the  learner  names  as  goals  for 
her/himself.  If  they  have  no  ideas,  suggest  goals 
based  on  their  responses  to  one  of  the  self-assess- 
ment tools. 

2.  Reading  Goals:  Make  goals  as  specific  as 
possible.  If  the  learner  has  a general  goal  such  as 
“reading  better”,  but  has  no  ideas  as  to  how  to  do  this, 
suggest  a goal  based  on  what  you  know  about  their 
reading  based  on  the  Miscue  analysis,  participation 


in  groups,  and  individual  conferences.  “Read  bet- 
ter” might  translate  into  working  on  a specific  read- 
ing strategy  like  using  prediction  and  contextual  cues 
to  make  sense  of  what  they  read.  “Reading  more” 
might  translate  into  choosing  a book  to  read  at  home 
and  making  a commitment  to  turn  off  the  TV  for  30 
minutes  a night  to  have  the  quiet  to  read  it. 

3.  Writing  Goals:  Make  these  goals  as  specific  as 
possible,  too.  Ask  the  writer  to  think  about  what’s 
good  about  their  writing,  what  they  like  about  it.  Tell 
them  about  some  positive  aspects  of  their  writing  that 
you  have  noticed  - content  and  process,  as  well  as 
mechanics.  Ask  the  writer  to  tell  you  some  things 
they  would  like  to  do  better  in  writing.  If  “spell 
better”  is  a goal,  help  the  learner  come  up  with 
specific  ways  they  can  work  on  their  spelling  and 
make  that  one  of  the  goals.  Try  to  make  some  of  the 
writing  goals  process  or  content  oriented  and  some 
of  them  skills  oriented. 

4.  Other  Goals:  These  may  be  interests  indicated 
on  the  Goals  Checklist  or  they  may  be  related  to 
meeting  the  learner’s  immediate  needs,  or  steps  to 
meeting  longer  term  goals,  such  as  getting  a driver’ s 
license.  In  class  and  outside  work  could  be  part  of  the 
plan.  Goals  may  also  come  out  of  work  in  options 
chosen  by  learners  over  eight  week  period,  such  as 
math,  computers,  or  helping  kids  with  homework. 


Notes 

1 All  of  the  materials  included  in  this  article,  as  well  as 

other  curriuculum  materials,  can  also  be  found  in  the 
ABE  0-4  (grade  level)  Curriculum  Kit  sponsored  by 
SABES. 

2 There  are  many  possibilities  for  teacher-made  assess- 

ment exercises.  This  is  only  one  example  of  an  exer- 
cise that  has  been  part  of  our  quick,  to  the  point,  but 
necessarily  superficial  assessment  of  potential 
learners'  ability  to  read  and  understand  a piece  of 
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connected  text.  It  helps  us  to  place  a learner  in  a very 
large  ballpark,  certainly  not  to  pinpoint  exactly 
where  s/he  is  sitting  or  exactly  which  reading  strate- 
gies s/he  has  strengths  and  weaknesses  in. 

3 Read/Write/Now  is  grateful  to  the  Adult  Literacy 

Evaluation  Project,  at  the  Center  for  Literacy  in 
Philadelphia,  the  source  of  the  original  goals  check- 
list that  we  used,  adapted  and  are  continually  revis- 
ing to  fit  the  needs  of  the  learners. 

4 See  Many  Literacies:  Modules  for  Training  Adult  Be - 

ginning  Readers  and  Tutors , "Life  Goals:  Maria's 
Story, " for  an  example  of  a group  activity  for  setting 
goals. 

5 Original  interview  developed  by  C.  Burke  (1987). 

6 Adapted  and  expanded  from  Sylvia  Greene's  "Writing 

and  Spelling  Progress  Sheet"  from  the  Basic  Liter - 
acyKix{ 1989). 
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"Down  and  Dirty"  Miscue  Analysis 


by  Lindy  Whiton 


Introduction 

Whole  language  is  based  on  the  theory  that 
readers  predict,  interact,  and  confirm  while  they  are 
reading.  They  use  letter-sound,  syntax,  ana  seman- 
tic (meaning)  cues  to  make  those  predictions  in  order 
to  understand  what  they  are  reading.  The  miscue 
analysis  measures  their  ability  to  use  these  cue  sys- 
tems and  strategies.  This  information  allows  the  in- 
structor to  develop  strategy  lessons  appropriate  for 
the  individual  reader. 

For  example,  a reader  may  score  85  percent 
acceptability  for  letter-sound  cues  and  20  percent 
grammatical  acceptability.  The  same  reader  may  be 
good  at  using  letter-sound  cues  to  predict,  but  not 
good  at  using  meaning  cues.  This  reader  leaves 
nonsense  words  in  the  passage  or  substitutes  one 
word  for  another  even  though  it  makes  no  sense  and 
obscure  the  meaning  of  the  passage.  This  reader 
might  be  given  strategy  lessons  that  would 
strengthen  her/his  usage  of  syntax  cues,  as  well  as 
ones  that  would  strengthen  semantic  strategies. 

Be  aware  that  the  miscue  analysis  is  not  a skills 
test.  It  evaluates  the  efficiency  of  the  strategies  that 
a reader  uses  and  it  indicates  where  help  is  needed. 
With  that  in  mind,  the  following  notes  outline  how  to 
do  a miscue  analysis: 

Materials 

1 .  A complete  text-one  that  is  slightly  challeng- 
ing to  the  reader  and  takes  approximately  15 
minutes  to  read 


2.  A copy  of  the  text  that  you  can  mark  on 

3.  A tape  recorder  and  tape 

4.  Miscue  analysis  form  ( see  copy  of  form  by 
B.  Sherman  & J.  DeLawter,  c.  1977,  appen- 
dix 13) 

Procedure 

Make  sure  the  tape  recorder  is  working.  Start  the 
tape.  First,  have  the  reader  read  the  entire  text. 
Keep  your  comments  supportive,  but  do  not  supply 
words;  suggest  that  s/he  “go  on,”  or  “skip  it.”  After 
s/he  has  finished  reading,  ask  for  a retelling.  Do  not 
give  any  prompts.  What  the  reader  finishes  retelling 
the  story,  you  can  that  ask  direct  questions,  such  as, 
Who  were  the  characters?  How  would  you  describe 
that  character?  What  do  you  think  the  author  was 
trying  to  tell  us?  Finally,  turn  off  the  tape,  thank  the 
reader  and  tell  her/him  s/he  may  go. 

Now  listen  to  the  tape,  marking  each  miscue  as 
you  go.  Number  them.  For  this  "down  and  dirty" 
miscue  analysis,  stop  marking  at  miscue  number  25. 
(In  a full  miscue  analysis,  you  would  mark  them  all.) 
However,  listen  to  the  entire  tape  once  to  get  an  idea 
of  the  reader’s  dialect,  repeated  miscues,  and  to  see 
how  many  non- word  miscues  appear.* 

The  miscues  to  be  marked  are: 

1)  If  the  reader  guesses  phonetically  and  there 
is  no  such  word,  write  the  phonetic  represen- 
tation above  the  word. 

2)  If  a word  is  replaced,  write  in  the  replace- 
ment. 
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3)  If  the  word  is  omitted,  circle  it. 

4)  If  a word  is  inserted,  put  a caret  in  the  place 
where  it  was  inserted. 

5)  If  the  miscue  is  corrected  by  the  reader, 
underline  the  miscue  and  draw  a line  in  front 
of  it,  placing  a circled  "C"  above  it. 

Once  you  have  marked  the  text,  you  are  now 
ready  to  fill  in  the  miscue  analysis  form.  Look  at 
each  miscue  individually  for  high  or  low  letter-sound 
relationship,  then  grammatical  acceptability  and 
whether  or  not  the  miscue  changes  the  meaning  of 
the  sentence  (sections  H-IV) . If  there  is  no  meaning 
change,  then  the  miscue  is  automatically  a strong 
miscue  and  gets  marked  within  the  pattern  section 
(V)  as  “strength” . If  there  is  a complete  change  in 
meaning,  it  is  automatically  a weak  miscue  and  gets 
marked  as  such . If  there  is  an  attempt  to  correct  the 
miscue  and  it  is  grammatically  acceptable,  there  is 
“some  strength”  to  the  miscue  and  it  should  be 
marked  as  such. 

Determine  the  percentages  for  each  grouping. 
What  you  are  looking  at  is  the  efficiency  with  which 
the  reader  uses  each  cue  system. 

Now  listen  to  the  retelling . For  each  element  of 
the  retelling  (eg.,  character  identification),  the 
reader  can  receive  a given  number  of  points  (section 
VI).  Use  your  judgement.  How  many  characters 
were  identified?  How  accurate  was  the  description 
of  each  character?  How  many  of  the  events  of  the 
story  did  the  reader  recall?  Did  the  reader  remember 
the  order  of  events?  Did  the  reader  understand  the 
plot?  Did  s/he  know  what  the  message  or  moral 
was?  And  finally,  did  s/he  bring  her/his  own  expe- 
rience into  the  retelling?  Add  up  the  points. 

Section  VII  is  important  too.  Repeated  miscues, 
such  as  quotation  marks,  or  -ing  or  -ed  endings,  may 
indicate  a particular  problem.  Completing  the  bar 
graph  summary  (VIII)  will  tell  you  where  the 
reader’s  strengths  and  weaknesses  are.  For  ex- 
ample, if  the  reader  gets  an  85  percent  for  sound- 


letter  relationships  and  a 10  percent  for  meaning, 
then  the  reader  may  need  to  learn  that  the  passage 
must  make  sense.  This  reader  may  also  not  know 
that  s/he  can  correct  miscues. 

By  reading  the  graph  and  understanding  the 
patterns,  individual  reading  strategy  lessons  can  be 
developed  for  that  individual  reader’s  needs.  For 
further  help  in  looking  at  Miscue  Analysis,  The 
Whole  Language  Evaluation  Book,  by  Goodman, 
Goodman  & Hood  (Heinemann)  is  helpful.  For  help 
in  developing  strategy  lessons  please  refer  to  Whole 
Language  Strategies  for  Secondary  Students,  edited 
by  Gilles,  Bixby,  Crowley,  Crenshaw,  Henrichs, 
Reynolds  & Pyle.  (Richard  C.  Owens). 


Note 

* Non-word  miscues:  this  one  is  especially  tale  telling,  for 
it  indicates  that  the  only  real  strategy  the  reader  is 
using  is  a letter-sound  strategy  and  that  other  strate- 
gies need  to  be  developed. 
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The  Education  Goals 
Assessment  Packet 


by  Martha  Germanowski 


fillip  he  Step-by-Step  Adult  Learner  Program  in 
g§  Amesbury  and  Haverhill  Massachusetts  is 
111  a grant-funded  program  for  homeless 
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adults.  The  ledincis,  men  aiiu  wunicii  age  ib-bj, 
range  in  skills  from  new  readers  to  high  school 
graduates.  They  come  to  the  program  primarily  from 
two  local  transitional  housing  programs:  Link 

House,  a live-in  program  for  men  with  a history  of 
substance  abuse,  and  Transitional  Housing  for 
Women,  a battered  women’s  shelter. 


Name_ 

Essential  Mathematics  for  Life 

Lifc^Skill  Questionnaire 

Here  is  a list  of  things  you  can  (Jo  with  math.  Put  an  “X* 
in  the  column  that  tells  how  you  feel  about  each  skill. 


Date 


.Aire: 

know 

how 

idy  ' 
L 

Want  to 
know 
right 
away 

Want  ro 
‘know 
some- 
time 

Don't 

care 

about  this 

Add  change 

Subtract  change 

Add  dollars  and  cents 

Make  change 

Estimate  dollar  total  while  shopping 

Find  amount  of  savings  with  sale  prices 

Find  unit  prices  to  get  (he  best  buy 

Figure  total  cost  on  a bill 

Figure  cost  of  a telephone  call 

Compare  meal  costs 

Find  S3les  tax 

Figure  total  cost  of  buying  on  an  installment  plan 

Find  interest  on  saved  or  borrowed  money 

Fieure  i-retlil  .-h-»ro*« 

Identifying  the  Need 

In  late  1990,  after  six  months  of  teaching  Step- 
by-Step  classes,  I identified  three  needs  to  address  in 
order  to  make  my  work  more  effective:  i)  I needed 
a more  systematic  way  to  learn  about  the  interests 
and  goals  of  my  adult  learners;  2)  I needed  to 
develop  individualized  education  plans  (IEPs)  for 
each  student;  and  3)  I needed  an  efficient  way  to 
record  the  on-going  progress  of  each  student.  I was 
not  content  with  our  program’s  brief  interview  and 
standardized  testing  as  the 
source  from  which  to  design 
class  materials  and  evaluate 
progress.  Those  methods  were 
convenient  but  severely  lim- 
ited. I wanted  a model  that 
measured  progress  in  a way  that 
both  teacher  and  student  would 
understand  and  find  relevant. 


FIGURE  1 


Devloping  the  Packet 
Through  a series  of  work- 
shops on  evaluation  and  assess- 
ment sponsored  by  SABES,  I 
began  to  investigate  various  in- 
take and  assessment  models 
used  in  other  Massachusetts 
adult  basic  education  programs 
( in  particular  the  goals  list  de- 
veloped at  the  Read/Write/ 
Now  program  and  published  in 
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Marilyn  Gillespie's  book,  Many  Literacies:  Mod- 
ules for  Training  Beginning  Readers  and  Tutors  ’). 
Then,  using  information  provided  at  Step-by-Step 
intake  interviews,  I compiled  a list  of  students’  inter- 
ests and  goals.  This  list  included,  “getting  my  GED,” 
“passing  a learner’s  permit  test,”  “reading  to  my 
children,”  “using  cooking  meausres,”  and  “filling 
out  a job  application.  ” To  this  list  I added  the  types 
of  materials  that  had  been  used  and  requested  by 
learners  over  the  first  six  months  of  my  class.  Eng- 
lish, math  and  life  skills  tasks  from  multi-level  ABE 
workbooks  such  as  Essential  Math  for  Life  (Con- 
temporary Books),  Let’s  Work  It  Out:  Topics  for 
Parents  (New  Readers  Press) , Master  Your  Money 
( Janus  Books),  and  Life  Skills  Reading  (Scott  Fores- 
man  and  Co.)  were  combined  with  the  original  list. 
Eventually  this  list  was  to  be  integrated  with  two 
other  models  and  modified  to  fit  our  program  ’ s par- 
ticular needs. 

Learner  Input 

Before  devising  the  final  goals  list  for  class- 
room use,  two  groups  of  about  five  learners  each, 
were  given  the  “Goals  List”  from  Many  Literacies 
and  the  “Life  Skills  Questionnaire”  from  Essential 
Mathematics  for  Life  (Scott  Foresman  and  Co.)  with 
one  modification:  the  column  heading  “like  to  know 
more”  was  added . (figure  1 ) Students  were  asked  to 
report  their  feelings  about  the  checklists.  They  re- 
sponded that  both  lists  had  interesting  items  and 
tilings  they  had  not  thought  of.  Some  learners  felt  that 
the  “Life  Skills  Questionnaire”  was  too  long,  while 
others  liked  the  variety.  Most  felt  that  the  “Goals 
List”  explanation  was  unclear,  but,  all  agreed  that  the 
print  was  easier  to  read  on  the  “Goals  List”  than  on 
the  “Life-Skills  Questionnaire.  ” 

Midway  through  this  investigation  with  learn- 
ers , I applied  to  S ABES  for  a mini-grant  to  continue 
this  assessment  work  in  greater  detail. 

A review  of  by  Step-by-Step  teachers  of  the  in- 
formation lead  to  the  conclusion  that  a checklist 


could  create  more  diverse  and  complete  responses 
than  a traditional  intake  form.  The  teachers  believed 
that  several  factors  should  be  carefully  attended  to 
when  designing  the  checklist:  Variety,  clarity  and 
format  are  important  to  create  interest  and  involve- 
ment in  the  survey  and  to  assure  that  learners  under- 
stand the  form  it.  Checklists  should  include  a means 
of  acknowledging  improvement,  advancement  or 
completion.  Students  should  be  given  options  to 
identify  and  choose  academic  goals  as  well  as  life 
skills.  Language  in  the  checklist  should  affirm  adult 
learners’  prior  knowledge,  be  sensitive  to  varying 
interest  levels,  and  avoid  negative  attitudes. 

Based  on  these  conclusions,  the  decision  was 
made  to  create  a checklist  of  interest  areas  from  all 
student  and  class  input  and  from  collected  published 
material.  The  checklist  would 

- be  printed  clearly  and  spaced  for  easy 
reading; 

- group  goals  according  to  interest  areas  and 
arrange  them  in  order  from  the  simplest  to 
the  most  complex  when  applicable; 

- list  life  skills  and  academic  skills  sepa- 
rately; 

- provide  a place  for  noting  completions  of 
tasks; 

- record  student  goals; 

- create  a daily  record  of  work  toward  goals 
started,  work  in  progress  and  work  com- 
pleted; 

- include  academic  skills  (math,  English, 

GED,  etc.) 

- be  worded  survey  in  a positive  manner.2 

A daily  log  was  designed  as  a way  to  have  a 
record  of  on-going  progress.  It  included:  goals  for 
the  day,  subjects  studied,  learner  comments  and 


teacher  commentsand  a plan  for  the  next  time. 

Our  first  Assessment  Packet  contained  these 
elements: 

- a daily  log  (appendix  1 6)) ; 

- a list  of  education  goals  (appendix  17)  with 
space  to  note  the  date  each  goal  is 
achieved; 

- a six  page  checklist  (appendix  18); 

- a signed  contract  as  required  by  our 
agency. 

This  version  of  the  Education  Goals  Assess- 
ment Packet  was  used  in  my  classroom  from  January 
to  March,  1991  and  distributed  at  SABES  staff  de- 
velopment meetings  throughout  the  state. 


Revisions 

In  April  a major  revision  was  made.  Classroom 
use  had  shown  that  students  rarely  went  back  to  the 
checklist  to  mark  completions  even  when  reminded. 
So,  the  checklist  was  revised  to  say  “Does  this 
subject  remind  you  of  anything  else  that  interests 
you?”  in  order  to  encourage  more  information  at 
intake.  The  “date  goals  achieved”  on  the  education 
goals  sheet  was  dropped  and  a monthly  review 
replaced  it  (appendix  19).  Along  with  the  daily  log, 
the  monthly  review  became  an  update  on  what  had 
been  learned  for  student  and  teacher  to  observe.  The 
review  included  re-reading  the  checklist  and  goals 
list,  reviewing  goals,  if  desired,  and  listing  advance- 
ments made  in  and  out  of  class. 

Atpresent,  no  further  changes  to  the  Education 
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Goals  Assessment  Packet  have  been  made.  How- 
ever, specific  academic  checklists  for  GED  Math 
have  been  added  for  students  who  are  doing  that 
preparation  (appendix  20).  Writing  and  English  lan- 
guage checklists  are  being  developed. 

Results 

Before  I began  using  the  Education  Goals 
Assessment  Packet,  I had  to  depend  on  time  consum- 
ing record  keeping  of  students’  work  and  progress, 
and  intake  forms  and  brief  conversations  as  the  only 
means  of  appraising  students'  interests  and  goals.  In 
addition,  students  were  minimally  involved  in  plan- 
ning and  goal-setting. 

Now  that  the  Packet  is  in  use,  many  changes 
have  occurred  in  the  classroom.  The  most  immediate 
change  is  that,  with  learners  keeping  their  own  rec- 
ord, there  is  much  more  time  for  me  to  give  individ- 
ual attention.  Through  the  Assessment  Packet,  stu- 
dents have  a more  complete  picture  of  their  class- 
work  and  progress.  After  discussing  their  choices 
with  the  teachers,  students  can  now  write  their  own 
goals  whichbecome  the  basis  for  their  individualized 
education  plans.  These  choices  give  students  op- 
tions, increasing  the  likelihood  that  theirs  will  be  a 
positive  learning  experience.  The  daily  log  is  a con- 
tinual reminder  of  progress  made  over  short  and  long 
periods.  It  has  become  a record  for  group  work  as 
well  as  class  development  (figure  2).  It  is  also  a 
document  of  our  curriculum  in  process,  a "retroactive 
syllabus." 

The  Education  Goals  Assessment  Packet  has 
been  distributed  to  the  Massachusetts  Homeless 
Education  Network,  Literacy  Volunteers  of  Massa- 
chusetts tutors,  and  presented  to  adult  educators  from 
Massachusetts  at  SABES  meetings  and  at  the  Liter- 
acy Volunteers  of  Massachusetts  Annual  Confer- 
ence. 

Some  programs  have  modified  it  to  fit  their 
learners'  capabilities.  Other  programs  have  used  it  as 
is.  Overall,  the  Packet  has  been  an  empowering  tool 


for  learners  as  well  as  teachers.  It  stimulates  adult 
learners  to  recognize  past  achievements  and  present 
possibilities.  Hie  daily  log  is  a simple  visual  record  of 
learner  progress  that  the  student,  teacher  or  tutor  can 
easily  understand. 


Reference/Note 

1 Gillespie,  Marilyn,  Many  Literacies:  Modules  for  Train- 

ing Adult  Beginning  Readers  and  Tutors  (Amherst: 
Center  for  International  Educaiton,  1990) 

2 They  were  first  changed  to  say,  “I  know  this,  I know 

some  of  this,  I would  like  to  know  more  now/later, 
not  interested  now.”  But,  before  being  distributed, 
the  heading  were  changed  to  say;  “I  know/do  this,  I 
would  like  to  know  more  now/later,  I understand  this 
and  I am  ready  for  the  next  step.” 


41 


Alternative  Assessment: 
An  Annotated  Bibliography  (excerpt) 


complied  by  Don  Robishaw 


From  the  Foreword 

Today  many  adult  educators  are  concerned 
about  mandates  from  federal, state  and  local  funding 
sources  that  call  for  standardized  testing  of  adults  in 
literacy  programs.  The  various  books,  dissertations, 
research  reports  and  articles  annotated  in  this  bibli- 
ography represent  a sample  of  case  studies,  descrip- 
tions of  alternative  approaches,  tools,  opinions  and 
arguments  in  defense  of  alternative  assessment 
procedures  that  better  serve  the  needs  of  adult 
learners. 

Traditional  standardized  quantitative  approaches 
to  assessment  in  adult  literacy  have  been  deemed 
inadequate  by  many  in  the  adult  literacy  field . Stan- 
dardized tests  often  close  off  or  lock  students  out  of 
opportunities  to  learn.  The  following  arguments 
have  been  used  in  defense  of  adult  educators  who 
believe  that  alternative  procedures  need  to  be  devel- 
oped to  better  serve  adult  learners: 

1.  Tests  designed  for  children  should  not  be 
used  with  adults. 

2 . Standardized  tests  do  not  measure  how  adults 

use  their  new  skills  in  real  life  experiences. 

3.  Elementary  school  grade  levels,  as  a form  of 
measurement  are  actually  degrading  to 
adults. 

4.  Adults  are  intimidated  by  testing  experiences. 

5 . Tests  remind  adults  of  their  past  failures. 

6.  Assessment  should  be  participatory. 


7.  Assessment  tools  should  be  designed  to  assist 
the  adult  learners  in  improving  their  new 
skills  and  to  help  teachers  set  up  structures 
for  improved  learning. 

Non-traditional,  holistic,  learner  centered  or  al- 
ternative approaches  to  assessment  contribute  to 
making  education  a different  and  more  positive 
experience  for  adults  than  the  negative  experience 
most  adults  in  ABE  programs  remember  as  kids.  The 
movement  towards  non  standardized  assessment  is  a 
major  step  in  the  right  direction  for  adult  education. 

Alternative  approaches  introduce  procedures 
that  enable  adult  literacy  students  to  evaluate  their 
own  experience  and  progress.  These  approaches 
also  help  students  view  their  own  learning  process  in 
reading  and  writing.  They  also  help  teachers  identify 
the  strategies  students  use  and  how  these  strategies 
change  as  they  progress  as  learners. 

The  main  issue  for  alternative  approaches  is  to 
make  the  assessment  process  participatory.  Creative 
tools  are  being  designed  to  assist  adult  learners  to 
assess  themselves  as  they  improve  their  skills  and  to 
help  adult  educators  set  up  structures  for  improving 
learning  opportunities.  These  new  and  innovative 
approaches  to  assessment  are  designed  to  be  an  on- 
going part  of  curriculum.  They  inform  the  develop- 
ment of  a curriculum  which  is  based  on  the  learners 
evolving  progress.  In  this  way  they  are  more  useful 
to  students  and  teachers  than  traditional  assessments 
at  the  completion  of  a course. 

Alternative  assessment  instruments  and  methods 
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are  used  by  adult  educators  to  collect  information 
about  student  knowledge,  skills  and  interests  to 
design  a learner-centered  curriculum.  It  is  the  opin- 
ion of  many  of  these  authors  that  assessment  should 
be  on-going  and  designed  to  be  part  of  the  curricu- 
lum, not  something  separate  or  added  on.  They 
combine  these  tools  and  methods  to  assist  the  learner 
in  progressing  in  adult  literacy  programs.  Examples 
of  some  of  these  tools  and  methods  include: 

- Learner  goals  checklists 

- Writing  progress  checklists 

- Reading  progress  checklists 

- Learner’ s writing  folders 

- Learner’ s journals 

- Learning  contracts 

- Notes  teachers  make  during  confer- 
ences 

- Learning  logs 

- Anecdotes 

- Record  keeping  devices 

To  support  and  enrich  teachers’  endeavors,  the 
Research  and  Design  unit  of  SABES  has  compiled 
this  first  edition  of  an  annotated  bibliography  on 
alternative  assessment  approaches  in  adult  literacy. 
This  bibliography  is  intended  to  be  a resource  to 
teachers  who  are  interested  in  learning  more  about 
the  topic  as  well  as  those  interested  in  using  the 
various  procedures  described  in  these  articles  in 
their  classrooms. 

Sample  Annotations 

022  ALBSU  (1990).  "Progress Profile."  Notting- 
ham, England:  Center  for  Research  into  Education 
of  Adults. 

Assessment  Adult  Literacy  Student  centered 


This  student  evaluation  profile  was  developed 
by  ALBSU  as  a model  to  illicit  and  maintain  student 
progress  in  adult  literacy  programs.  As  a framework 
for  student  assessment  in  adult  literacy  it  reflects  a 
student-centered  approach. 

The  progress  profile  initially  tries  to  get  at  student 
goals  by  asking  the  question  “Where  do  I want  to 
go?”  Students  decide  on  short  term  learning  goals  to 
work  on . these  are  goals  that  progress  can  be  made 
on  during  a maximum  of  forty  hours  of  work.  From 
this  point  students  and  tutors  are  expected  to  con- 
tinue on  with  the  following  questions  in  order  to 
complete  the  cycle  and  then  to  start  the  cycle  over 
again  with  the  first  question  or  to  continue  working 
on  the  students  present  concern  until  they  feel  con- 
fident about  their  new  learning,  the  remaining  four 
questions  are: 

2.  What  do  I need  to  leam? 

3.  How  am  I going  to  get  there? 

4.  How  far  have  I got? 

5.  Where  to  next? 


023  Wolf,  M.  (1988).  "Towards  a Model  of  Alter- 
native Assessment:  A Progress  Report."  Bronx,  NY: 
Institute  for  Literacy  Studies,  Lehman  College. 

Action  research  Teacher/Learner  participation 

This  short  term  participatory  research  project  is  a 
case  study  of  an  experiment  involving  teachers  and 
learners  from  an  adult  literacy  program  working 
together  to  develop  assessment  tools  for  new  stu- 
dents. The  project  was  initiated  because  of  the 
teachers’  expressed  need  for  better  ways  to  assess 
the  reading  and  writing  abilities  of  adult  students. 
The  three  components  of  the  project  were:  1)  the 
selection  of  a research  team  including  both  teachers 
and  learners;  2)  a seminar  which  met  for  five  two- 
hour  sessions;  and  3)  two  sets  of  interviews  of  stu- 
dents by  student/researchers. 


The  article  might  have  been  more  helpful  had 
details  on  both  the  seminars  and  the  interview  been 
included.  Specifically,  the  lists  of  interview  ques- 
tions generated  in  the  seminars  as  well  as  the  ones 
finally  chosen  to  be  used  should  have  been  given. 

021  Goodman,  K.S.,  Goodman,  Y.  &Hood,  W.J. 
(1989).  The  Whole  Language  Evaluation  Book. 
Portsmouth,  N.H. : Heinemann. 

Whole  Language  Evaluation  Curriculum 

This  hock  contains  a chapter  which  centers  on 
evaluation  in  the  whole  language  classroom. 
Goodman  maintains  that  evaluation  is  part  of  the 
curriculum  and,  therefore  is  integral  to  the  teaching/ 
learning  process.  She  underscores  the  importance  of 
building  a professional  sense  about  evaluation  in 
language  teaching,  including  knowing  about  lan- 
guage and  learning  and  developing  intuition  about 
one’s  work.  Goodman  refers  to  the  “double 
agenda”  of  evaluation:  the  teacher’s  evaluation  and 
the  student’s  learning  and  emphasizes  that  evalu- 
ation is  an  on-going  process  built  into  the  everyday 
plans  of  the  teacher.  This  chapter  concludes  with  a 
discussion  of  observing,  interacting,  and  analyzing 
skills  and  of  the  importance  of  self-evaluation  as  part 
of  a teacher’s  professional  development. 

This  book  is  a primer  on  whole  language  teach- 
ing, an  introduction  to  the  field  for  the  uninitiated.  As 
such,  the  well-experienced  language  teacher  or  the 
person  well-versed  in  evaluation  will  find  this  chap- 
ter by  Goodman  rather  shallow  and  uninspiring.  It 
discusses  evaluation  in  a cursory  manna- and  fails  to 
address  many  of  the  complex  issues  involved. 


several  people— from  the  development  of  the  initial 
idea,  to  the  search  and  collection  of  books  and  ar- 
ticles. I am  indebted  to  the  following  people  for  their 
assistance  in  this  project:  Loren  McGrail  of  World 
Education/S  ABES  who  supplied  me  with  many  of 
these  resources  and  Joan  Dixon,  Literacy  Support 
Initiative  Coordinator.  In  addition  the  following 
people  acted  as  reviewers:  Janet  Kelly,  director  of 
the  Read/Write/Now  program,  Janet  Isserlis  of  the 
International  Institute  of  Rhode  Island  and  the  fol- 
lowing U.  Mass  graduate  students:  Michele  M. 
Sedor,  Barbara  Huff,  Susan  Schellenberger,  Haleh 
Arbab,  Keyvan  Kabastioun,  and  Ed  Graybill. 

The  complete  annotated  bibliography  contains  51  entries 
and  will  be  updated  on  an  ongoing  basis.  Contribu- 
tions to  the  collections  are  welcome.  Each  regional 
center  has  a collection  of  these  articles.  Please 
contact  your  Regional  S ABES  Coordinator  or  Loren 
McGrail  at  World  Education  if  you  would  like  to 
annotate  some  of  the  collected  articles  or  add  new 
ones. 


Notes 


As  with  many  works  which  assemble  a collection  of  read- 
ings this  bibliography  is  the  result  of  the  efforts  of 
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Appendix  2 


An  overview  of  the  assessment  process  and  tools  used  in  the  Read/ 

Write/Now  Adult  Learning  Center 


Initial  Assessment  (before  learner  enters  program): 

Initial  Screening  & Placement  Interview, 

with  Slosson  and  teacher-made  comprehension  exercise 


Early  Assessment  (first  3 weeks  of  class  cycle): 
Goals  Checklist 

Modified  Burke  Reading  Interview 
Looking  at  Your  Own  Reading  Behavior 
Miscue  Analysis 
Reading  Progress  Checklist 
Writing  Progress  Checklist 
Learning  Contract 


Ongoing  Assessment  (done  regularly  or  as  needed  throughout  cycle): 
Miscue  Analysis 
Book  List 

Reading  Conference  Record 

Teacher’s  Log 

Learner’s  Log 

Math  Activities  Record 

Writing  Conference  Record 

Review  Writing  folder,  dialogue  journal 

Review  and  revise  Learning  Contract 


Then  & Now  Assessment  (using  same  tool  again  or  new  tool  to  review  progress):* 

Writing  Progress  Checklist 

Reading  Progress  Checklist 

Modified  Burke  Reading  Interview 

Looking  At  Your  Own  Reading  Behavior 

Slosson  Post-test,  if  applicable 

Learner’s  Self-Evaluation  Questionnaire 

* A final  review  of  learner’s  assessment  portfolio,  which  would  include  samples  of  writing,  materials 
for  initial  and  ongoing  assessment,  goals  list,  learning  contracts,  etc. , would  be  part  of  a “then  and  now” 
assessment  conference  with  learner  at  the  end  of  a cycle. 


Appendix  3 


READ/WRITE/NOW  SCREENING  & PLACEMENT  INTERVIEW 
Name:  Interviewer:  ^ Date: 


1.  What  is  your  most  important  reason  for  wanting  to  learn  to  read  and  write  better? 


2.  What  are  some  things  that  you  want  to  do  that  being  able  to  read  and  write  better  will 
help  you  to  do? 


3.  Do  you  have  any  children  or  grandchildren  in  your  life?  If  so,  are  they  school  age? 
What  are  their  ages  and  grades  in  school? 


4.  Do  you  have  a job  now?  If  so,  what  kind  of  work  do  you  do?  If  not,  what  kinds  of  jobs 
have  you  had?  Do  you  need  to  use  reading  and  writing  at  work? 


5.  Do  you  do  any  reading  on  your  own  now?  Do  you  read  at  home,  in  stores,  at  work,  on 
the  street?  If  so,  what  kinds  of  things  do  you  read?  (Examples:  TV  schedule,  mail,  news- 
papers, childrens’  books,  labels,  street  signs...) 


6.  Do  you  do  any  writing  on  your  own  now?  If  so,  what  kinds  of  things  do  you  write? 
(Examples:  signs  name,  forms,  shopping  lists,  checks  or  money  orders,  letters  and  notes...) 


7.  Tell  me  something  about  your  school  years;  where  did  you  grow  up?  Did  you  go  to 
school  there?  What  was  it  like  for  you?  What  grade  did  you  finish? 


7a.  (For  non-native  speakers  of  English  and  immigrants)  What  is  your  first  language? 
WTien  did  you  come  to  the  U.S.?  Did  you  have  the  chance  to  go  to  school  in  your  native 
country?  If  so,  how  many  years?  Do  you  read  and  write  your  first  language? 


8.  Have  you  ever  been  to  classes  for  adult  education  before?  If  so,  where  and  when? 
What  was  it  like  for  you? 


9.  Have  you  ever  used  a computer  before?  How  do  you  feel  about  learning  something 
about  using  computers? 


10.  Some  adults  really  want  to  come  to  class  and  improve  their  reading  and  writing,  but 
find  that  things  get  in  their  way.  Some  of  the  things  that  come  up  are:  problems  with  trans- 
portation, childcare,  having  to  take  care  of  other  people  in  their  families  when  they  are 
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sick  or  need  them,  getting  sick  a lot,  having  problems  with  drugs  or  drinking,  too  much 
going  on  at  home,  working  long  hours...  What  kinds  of  things  do  you  think  might  make  it 
hard  for  you  to  come  to  study  and  get  to  class? 


10a.  What  do  you  think  you  could  do  to  make  it  easier  for  yourself  to  study  and  get  to  all  of 
the  classes? 


10b.  What  can  we  do  to  make  it  easier  for  you  to  study  and  come  to  all  of  the  classes? 
Examples:  help  figure  out  a bus  route,  find  information  on  childcare  providers,  fill  out 
forms... 


11.  We  have  a policy  against  drug  and  alcohol  use  by  students  and  teachers,  partly  because 
we  know  that  no  one  can  learn  or  teach  much  when  they  are  under  the  influence  of  either 
one.  If' you  are  invited  to  participate  in  learning  at  this  center,  do  you  agree  that  you  will 
not  come  here  under  the  influence  of  drugs  or  alcohol? 


Signed: 


Signed: 


0 
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Appendix  4 


READ/WRITE/NOW  SILENT  OR  ORAL  READING  COMPREHENSION 
QUICK  ASSESSMENT  EXERCISE 


Reader:__ Staff: Date: 

(Write  reader’s  responses  or  significant  points  of  retelling  on  this  sheet.  Use  other  side  if  needed.) 

Introduction:  This  is  the  first  page  from  a book  called,  The  Door  is  Open.  It’s  about  a 
woman  who  decided  to  go  back  to  school  as  an  adult  after  being  away  from  it  a long  time. 
Her  name  is  Lina.  Take  some  time  to  try  reading  this  to  yourself  silently.  After  you  finish, 
we  will  talk  about  what  you  read. 

* If  the  person  being  interviewed  indicates  that  they  can’ t read  it  silently  but  would  like  to  try  it  orally, 
that’s  fine.  If  the  reader  needs  help  in  getting  through  the  oral  reading,  help  as  needed  and  make  note  of 
miscues  made  that  change  the  meaning  of  the  text  and  are  not  self-corrected.  (Example:  Text  says:  “She 
had  wanted  to  return  to  school  for  many  years.  ” If  reader  miscues:  “She  had  wanted  to  read  the  school 
for  money  yes” , without  self-correcting,  make  note  of  it  on  your  copy  of  the  text.)  A reader  like  this  may 
not  know  that  reading  makes  sense,  is  relying  too  much  on  graphophonic  cues  and  will  need  help  devel- 
oping other  strategies. 

After  silent  reading:  How  was  that  for  you  to  read?  Hard  to  read,  just  about  right,  or  easy 
to  read?  What  made  it  hard  for  you?  (or  just  right,  or  easy) 


After  silent  or  oral  reading:  Ask  the  reader  to  tell  you  what  they  remember  about  the  read- 
ing. If  questions  are  needed  to  guide  the  retelling,  use  some  or  all  of  these  questions: 


1 . What  did  Lina  want  to  do  for  so  many  years? 

2.  How  long  had  she  been  out  of  school? 

3.  How  did  she  feel  about  going  into  the  school  for  the  first  time? 

4.  This  is  not  in  the  story,  but  I would  like  to  know  what  you  think:  Why  do  you  think  Lina  decided  to  go 
back  to  school  after  all  those  years? 

COMMENTS: 


0 
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Appendix  5 


Something  We're  Working  On 

Using  learner-written  pieces  for  reading  assessment:  There  are  many  pieces  of  writing  by 
other  learners,  such  as  those  published  in  Voices  magazine  which  can  be  used  in  a similar  way  to  get 
both  a sense  of  a learner’s  reading  ability  and  to  stimulate  feelings  of  identification  and  solidarity  with  the 
writers’  words  they  are  reading.  It  can  be  a nice  way  to  introduce  a new  student  to  a program  where  they 
will  be  seen  as  creators  of  culture,  and  not  just  consumers  of  it. 

In  using  pieces  from  Voices,  for  instance,  learners  can  be  offered  a choice  among  several  pieces. 
They  might  choose  one  or  more  to  read  and  talk  over  with  the  interviewer. 

The  pieces  should  be  chosen  for  their  natural  language,  potential  interest  and  anticipated  meaning  for 
the  reader,  and  varying  levels  of  difficulty.  Pieces  from  the  First  Words  or  New  Words  sections  of  the 
magazine  are  usually  very  accessible  to  basic  and  beginning  readers.  There  may  only  be  two  or  more 
sentences  of  connected  text.  Sentences  are  short,  simple,  and  often  have  repeated  phrases  or  patterns. 
See  “Hope”  by  Julia  Williams  and  “Good  Worker”  by  Jasper  Bullock  for  examples.  You  can 
include  some  more  difficult  pieces  from  the-  same  section  of  the  magazine.  See  “Me  First”  and 
“Spending  Time”  by  Mary  Boyd  for  examples  of  moderately  more  difficult  pieces.  By  “moderately 
more  difficult”  I mean  longer  pieces  that  still  use  short  sentences,  natural  language  and  contain  fewer 
repetitive  patterns.  More  difficult  pieces  such  as  “A  Fear  That  I Have”  by  Kelly  Young  or  “To 
Past  Teachers”  by  Hector  Diaz,  use  more  multisyllabic  words,  idiomatic  expressions  and  descriptive 
language  that  makes  the  pieces  interesting  to  the  reader  and  also  more  difficult. 

The  level  of  difficulty  of  the  pieces  used  in  this  reading  assessment  activity  can  be  assigned  grade 
levels,  if  needed,  by  using  a readability  formula,  such  as  the  Frye  formula,  beforehand,  and  offering 
pieces  of  varying  “grade  levels” . We  have  been  thinking  of  using  this  approach  for  assessment  of 
connected  reading,  using  different  samples  for  the  pre-  and  post-tests.  We  think  it  will  give  us  a more 
complete  idea  of  a learner’s  ability  to  read  and  understand  what  they  read  and  also  reflect  some  of  the 
growth  our  lower  level  learners  make  in  reading  that  doesn’t  show  up  on  the  Slosson.  It  also  seems  like 
it  would  make  a less  alienating  “testing”  situation  in  that  it  uses  materials  written  by  other  learners  and  it 
allows  the  learner  to  choose  the  readings,  based  on  interest  as  well  as  whether  they  are  accessible  to 
them. 

After  silent  reading  one  piece  the  learner  selects,  ask  them  to  tell  you  about  the  piece.  If  the  reader’s 
retelling  lacks  details,  ask  some  questions  designed  to  help  the  reader  recall,  relate  to,  and  interpret  what 
they  read.  See  copies  of  the  pieces  above  and  Sample  Questions  that  could  be  used  with  them.  Some 
of  the  sample  questions  are  to  help  the  reader  recall  and  retell  meaningful  details  of  the  pieces  and  others 
are  meant  to  encourage  the  reader  to  interpret  and  respond  to  the  content  of  the  reading.  If  the  reader 
can’t  remember  what  they  read  or  didn’t  understand  it  well  enough  to  talk  about,  you  could  offer  to  read 
it  aloud  and  then  discuss  it  together.  If  the  reader  wants  to  try  another  piece,  encourage  them  to  try  one 
of  the  easier  ones,  and  see  how  that  goes.  The  reader  can  stop  at  after  any  piece  that  they  want  to. 

The  results  of  this  informal  assessment  activity  are  likely  to  be:  gaining  some  insight  into  the  reader’s 
ability  to  recall  and  respond  to  texts  of  various  levels  of  difficulty.  It  may  also  help  to  determine  what 
levels  of  text  a reader  can  handle  independently. 
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SAMPLE  QUESTIONS  FOR  READING  ASSESSMENT  USING  STORIES  FROM 

VOICES 

Reading:  “Good  Worker”  by  Jasper  Bullock 


Questions: 

1.  What  kind  of  work  does  the  writer  say  he  does?  (for  recall) 

2.  Does  his  picture  tell  you  anything  about  what  work  he  does?  (for  recall  & interpretation) 

3.  Why  do  you  think  he  says  he  is  a good  worker?  (for  interpretation) 

4.  What  do  you  think  makes  a person  a good  worker?  (for  interpretation) 


Reading:  “Hope”  by  Julia  Williams 


Questions: 

1.  What  does  the  writer  hope  to  do?  What  else?  (for  recall) 

2.  Why  did  she  call  her  story  “Hope”?  (for  interpretation) 

3.  If  you  were  to  write  a story  about  your  hopes,  what  would  it  be  like?  (interpretation) 


Reading:  “Me  First”  by  Mary  Boyd 


Questions: 

1.  Why  do  you  think  Mary  Boyd  called  her  piece  “Me  First”  ? (for  recall  & interpretation) 

2.  What  are  some  of  the  things  she  said  she  would  like  to  spend  money  on  if  she  had  it?  (for  recall) 

3.  What  do  you  think  of  the  things  she  wanted  to  do?  (for  interpretation) 

4.  What  would  you  do  if  you  had  a lot  of  money?  (for  interpretation) 


Reading:  “To  Past  Teachers”  by  Hector  Diaz 


Questions: 

1.  How  did  he  feel  about  the  teachers  he  had  known  in  the  past?  (for  recall) 

2.  What  did  he  think  of  the  way  his  teachers  taught?  (for  recall) 

3.  Why  did  he  get  passed  from  grade  to  grade?  (for  recall  & interpretation) 

4.  Did  you  ever  have  any  teachers  like  the  ones  Hector  writes  about?  (for  interpretation) 


Reading:  “A  Fear  That  I Have”  by  Kelly  Young 


Questions: 

1.  What  is  the  writer  afraid  of?  (for  recall) 

2.  How  did  she  cover  up  her  problems  with  reading  and  writing?  (for  recall) 

3.  What  happened  with  her  son  and  the  teddy  bear  that  helped  her  make  a change?  (for  recall  & 
interpretation) 

4.  Why  do  you  think  she  decided  to  get  help  with  her  reading  and  writing?  (for  interpretation) 


Appendix  6 


"Keep  a record  from  year  to  year"  SLOSSON  ORAL  REAPING  TEST  (SORT) 


NAM& 


AGE- 


Las 

r nor 

Ml  oou 

List  P (2o> 

List  1 (40) 

List  2 (60) 

1.  see 

1 . with 

1 . game 

2.  look 

2 . friends 

2 . hide 

3.  mother 

3.  came 

3 . grass 

4.  little 

4.  horse 

4.  across 

5.  here 

5 . ride 

5 . around 

6.  can 

6.  under 

6 . breakfast 

7 . want 

7 . was 

7 . field 

8.  come 

8 . what 

8 . large 

9.  one 

9 . bump 

9 . better 

10.  baby 

10 . live 

10 . suddenly 

11.  three 

11.  very 

1 l . happen 

12.  run 

1 

12  puppy 

12.  farmer 

13.  jump 

13.  dark 

13.  river 

14.  down 

14.  first 

14.  lunch 

15.  is 

15.  wish 

15.  sheep 

16.  up 

16.  basket 

16.  hope 

17.  make 

17 . food 

17.  forest 

18.  ball 

18.  road 

18.  stars 

19.  help 

19.  hill 

19.  heavy 

20.  play 

20 . along 

20.  station 

DATE- 


READING 

LEVEL 


SCHOOL 

GRADE 


SCHOOL- 


EXAMINERr- 


List  3 (80) 

List  4 (ioo) 

1 safe 

1 harness 

1 

2 against 

2 price 

2 

3 smash 

3 flakes 

3 

4 reward 

4 silence 

4 

5 evening 

5 develop 

5 

6 stream 

6 promptly 

6 

7 empty 

7 serious 

< 

D stone 

D courage 

o 

9 grove 

9 forehead 

9 

10  desire 

10  distant 

10 

1 1 ocean 

1 1 anger 

11 

12  bench 

12  vacant 

12 

13  damp 

13  appearance 

13 

14  timid 

14  speechless 

14 

15  perform 

15  region 

15 

16  destroy 

16  slumber 

16 

17  delicious 

17  future 

17 

18  hunger 

18  claimed 

18 

19  excuse 

19  common 

19 

20  understood 

20  dainty 

20 

List  5 mm 

cushion 
2 generally 
extended 
custom 
tailor 
haze 
gracious 
dignity 
terrace 


marriage 


obedient 


presence 

merchant 


List  6 (140) 

1 

installed 

2 

importance 

3 

medicine 

4 

rebellion 

5 

infected 

6 

responsible 

7 

liquid 

8 

tremendous 

9 

customary 

10 

malicious 

11 

spectacular 

12 

inventory 

13 

yearning 

14 

imaginary 

15 

consequently 

16 

excellence 

17 

dungeon 

18 

detained 

19 

abundant 

20 

compliments 

List  7 l i6o) 

1 

administer 

2 

tremor 

3 

environment 

4 

counterfeit 

5 

crisis 

6 

industrious 

7 

approximate 

8 

society 

9 

architecture 

10 

malignant 

11 

pensive 

12 

standardize 

13 

exhausted 

14 

reminiscence 

15 

intricate 

16 

contemporary 

17 

attentively 

18 

compassionate 

19 

complexion 

20 

continuously 

List  8 (ia<n 

1 prairies 

2 evident 

3 nucleus 

4 antique 

5 twilight 

6 memorandum 

7 whimsical 

0 proportional 

9 intangible 

10  formulated 

11  articulate 

12  deprecate 

13  remarkably 

14  contrasting 

15  irrelevance 

16  supplement 

17  inducement 
10  nonchalant 

19  exuberant 

20  grotesque 


_ List  9*12  ( sntn 

1 traverse 

2 affable 

3 compressible 

4 excruciating 

5 pandemonium 

6 scrupulous 

7 primordial 

<>  chastisement 
9 sojourn 

10  panorama 

11  facsimile 

12  auspicious 

13  contraband 

14  envisage 

15  futility 

16  enamoured 

17  gustatory 
10  decipher 

19  inadequacy 

20  simultaneous 


SCORE 

List 

P 

List 

L 

List 

2_ 

List 

* 

List 

A 

List 

List 

fi 

List 

7 

List 

8 

List 

a-12 

Raw 

Score 

(Total  number  of 
correct  wo  rd  s 
including  the 
words  below 
starting  1 eve  1.) 


(Any  specific  coaching  on  these  particular  words. will  naturally  reduce  the  validity  of  this  test.) 
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Takes  about  3 minutes 

SLOSSON  ORAL  READING  TEST  (SORT) 

Keep  this  test  in 

to  give  and  to  score. 

Copyright  © 1963,  Richard  L.  Slosson,  M.  A. 

safe  d rawer  or  file. 

This  Oral  Heading  Test  is  to  be  given  individually  and  is  based  on  the  ability  to  pronounce  words 
at  different  levels  of  difficulty.  The  words  have  been  taken  from  standardized  school  readers  and  the 
Heading  Level  obtained  from  testing  represents  median  or  standardized  school  achievement.  A correlation 
of  .96  (variability  on  a group  of  108  children  from  first  grade  thru  high  school:  Gray  Mean  = 5.0.  SORT 

Mean  = 5.0.  Gray  S.  D.  = 2.0.  SORT  S.D.  = 2.3)  * as  obtained  with  the  Standardized  Oral  Reading  Paragraphs 
by  Wiliam  S.  Gray , published  by  The  Bobbs^Merril  Company . Inc..  Indianapolis.  Indiana.  Permission  to 
use  this  test  by  Gray  for  purposes  of  validation  is  deeply  appreciated. 

A reliability  coefficient  o f .99  ( t est-^  ret  est  interval  of  one  week)  shows  that  this  Oral  Reading  Test 
can  be  used  at  frequent  intervals  to  measure  a child*s  progress  in  reading , providing  no  specific  coaching 
with  these  particular  words  has  been  given.  Such  periodic  testing  can  be  highly  motivating. 

PI RECTIONS 


1.  Allow  the  child  to  read  from  one  sheet  while 

you  keep  score  on  another.  At  the  start,  say  the 
following:  "/  want  to  see  how  many  of  these  words 

you  can  read.  Please  begin  here  and  read  each 
word  aloud  as  carefully  as  you  can.  " (Indicate  at 
what  list  to  start.  ) "tohen  you  come  to  a difficult 
word . do  the  best  you  can  and  if  you  can't  read 
it,  say  * blank * and  go  on  to  the  next  one.  " 

2.  Start  a child  with  a list  where  you  think  he 
can  pronounce  all  20  words  in  that  one  list  cor- 
rectly. Note  that  each  list  of  words  is  graded. 
List  P (primer)  is  for  the  first  few  months  of 
first  grade.  List  1 is  for  the  balance  of  first 
grade,  List  2 is  for  second  grade,  etc.  If  the 
starting  list  is  too  difficult  and  the  child  makes 
even  one  mistake,  go  back  until  you  reach  an  easier 
list  where  he  can  pronounce  all  20  words  correctly. 

3.  After  you  have  found  the  starting  list,  go  on 
into  more  advanced  1 ists  until  you  find  the  stopping 
list,  where  he  mispronounces  or  is  unable  to  read 
all  20  words.  When  you  reach  a point  where  the  words 
become  very  difficult,  say:  'took  quickly  down  th/s 
list  and  read  the  words  you  think  you  know.  " 

4.  When  a child  reads  very  slowly  and  takes  more 
than  5 seconds  on  each  and  every  word,  move  him 
along  by  saying  the  “blank”  for  him.  Or  call  out 
the  number  of  the  word  at  a rate  of  about  5 seconds 


each.  Still  another  plan  is  to  use  a small  card  or 
piece  of  paper,  covering  up  a word  after  a 5 second 
exposure,  forcing  him  on  to  the  next  word. 

5.  ODunt  as  an  error  each  mispronounced  or  omitted 
word  as  well  as  a word  which  takes  more  than  about 
5 seconds  to  pronounce.  (If  a child  has  a speech 
defect  such  as  a stutter,  disregard  the  5 second 
interval  and  allow  as  much  time  as  necessary. ) 
Count  it  an  error  when  a child  is  uncertain  about 
a word  and  gives  more  than  one  pronunciation,  even 
though  one  of  them  may  have  been  correct.  Be  par- 
ticularly careful  about  scoring  the  word  endings 
as  they  must  be  absolutely  correct.  Keep  score  by 
putting  a check  mark  (J  ) after  each  error  or  a 
plus  sign  (4-)  after  each  correct  word.  Enter  the 
number  of  correct  words  at  the  bottom  of  each  list 
as  you  go  along.  An  analysis  of  scatter  on  the 
test,  as  well  as  an  analysis  of  the  types  of 
errors  made,  will  indicate  areas  of  weakness. 

6-  To  find  a child's  raw  score  for  reading,  count 
the  total  number  of  words  he  was  able  to  pronounce 
correctly  in  all  lists  and  add  the  words  below  the 
starting  list  for  which  he  automatically  receives 
credit.  To  obtain  the  Reading  Level,  look  up  the 
value  of  this  raw  score  in  Table  1 below.  A simple 
way  to  determine  the  Reading  Level  is  to  take  half 
the  raw  score.  For  example,  if  the  raw  score  were 
46,  half  of  this  number  would  be  23  and  the  Reading 
Level  would  be  2.3  or  the  3rd  month  of  2nd  grade. 


TABLE  I 


CHANGING  THE  RAW  SCORE  TO  READING  LEVEL 

(Reading  Grade  Level  is  given  in  years  and  months.  For  example,  5.2  means  the  2nd  month  of  5th  grade.  ) 


SCORE 

GRADE 

SCORE 

GRADE 

SCORE 

GRADE 

SCORE 

GRADE 

SCORE 

GRADE 

SCORE 

GRADE 

SCORE 

GRADE 

0-1 

0.0 

26r27 

1 .3 

52-53 

2.6 

78-79 

3.9 

104-105 

5.2 

130-13! 

6.5 

156-157 

7.8 

2-3 

0.1 

28-29 

1.4 

54-55 

2.7 

80-81 

4.0 

106-107 

5.3 

132-133 

6.6 

158-159 

7.9 

4-5 

0.2 

30-31 

1 .5 

56-57 

2.8 

82-83 

4.1 

108-109 

5.4 

134-135 

6.7 

160-161 

8.0 

6-7 

0.3 

32-33 

1.6 

58-59 

2.9 

84-85 

4.2 

110-111 

5.5 

136-137 

6.8 

162-163 

8.1 

8-9  ' 

0.4 

34-35 

1.7 

60-61 

3.0 

86-87 

4.3 

1 12-1 13 

5.6 

138-139 

6.9 

164-165 

8.2 

10-1  I 

0.5 

36-37 

1.8 

62-63 

3.1 

88-89 

4.4 

1 14-1 15 

5.7 

140-  >4  1 

7.0 

166-167 

8.3 

12-13 

0.6 

38-39 

1.9 

64-65 

3.2 

90-91 

4.5 

1 16-1 17 

5.8 

142-143 

7.1 

168-169 

8.4 

14-15 

0.7 

40-41 

2.0 

66-67 

3.3 

92-93 

4.6 

118-119 

5.9 

144-145 

7.2 

170-171 

8.5 

16-17 

0.8 

42-43 

2.1 

68-69 

3.4 

94-95 

4.7 

120-121 

6.0 

146-147 

7.3 

172-173 

8.6 

18-19 

0.9 

44-45 

2.2 

70-71 

3.5 

96-97 

4.8 

122-123 

6.1 

148-149 

7.4 

174-175 

8.7 

20-21 

1.0 

46-47 

2.3 

72-73 

3.6 

98-99 

4.9 

124-125 

6.2 

150-151 

7.5 

176-177 

8.8 

22-23 

.1.1 

48-49 

2.4 

74-75 

3.7 

100-101 

5.0 

126-127 

6.3 

152-153 

7.6 

178-179 

8.9 

24-25 

1.2 

50-51 

2.5 

76-77 

3.8 

102-103 

5.1 

128-129 

6.4 

154-155 

7.7 

180-200 

9*  12 
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Appendix  7 


WRITING  SAMPLE 


Name:  Date: 


Introduction:  Write  as  much  as  you  can  and  take  as  long  as  you  need  to.  You  don’t  have  to 
fill  up  the  whole  page,  but  you  can  if  you  want  to.  Don’t  worry  about  spelling  or  handwrit- 
ing. Just  write  what  you  want  to  say.  Sometimes  people  like  to  write  about  some  of  their 
reasons  for  wanting  to  read  and  write  better.  You  can  write  about  anything  you  want  to. 


Appendix  8 


READ/WRITE/NOW  GOALS  LIST 


Name: 


Date: 


Pergpnal  Owls;  Can  Do  Work  On  No 

1.  Read/write  address 

2.  Write  a shopping  list 

3.  Write  checks  

4.  Read  bills  

5.  Use  the  phone  book 

6.  Read  a menu 

7.  Read/write  recipes 

8.  Take  the  driver’s  test 

9.  Read  leases  or  contracts 

10.  Read/write  letters  or  notes  

11.  Improve  math  skills 

12.  Learn  to  use  computer 

13.  Read  a newspaper  (what  parts?) 

14.  Read  magazines  (which  ones?) 

15.  Read  labels  & signs 

16.  Please  check  the  subjects  you 
like  to  read  about: 


histoiy 

politics 

science 

love,romance 

adventure 

crime 

occult 

lives  of  famous  people 


sports  (which  ones?) 

humor 

"how-to-do-it"  books 

plays 

poetry 


science  fiction 
other: 


17.  Read  maps  & write  directions 

18.  Cursive  writing 

19.  Improve  printing 

20.  Write  your  life  stoiy 

21.  Write  short  stories/poems 
Family; 

1.  Read  to  children 

2.  Help  children  with  homework 

3.  Read/write  notes  to  childrens’  school 

4.  Read/write  names  of  family 
Community; 

1.  Church  reading 

2.  Voting  registration/information 

3.  Find  out  more  about  how  the  government  works 

4.  Join  a group  to  solve  a problem 
Work; 

1.  Fill  out  job  applications 

2.  Write  resumes  & cover  letters 

3.  Read/write  telephone  messages 

4.  Read/write  job  instructions 

5.  Read  to  learn  about  other  jobs 

6.  Read/write  memos 

7.  Fill  out  order  forms/reports 

8.  Take  notes  at  work  meetings 


Education; 

Here  are  some  educational  goals.  Write  "yes"  if  you  are  interested  and  "no"  if  you’re  not; 

1.  Attend  a job  training  program  (what  kind?) 

2.  Attend  classes  to  learn  something  new  (crafts,  self-improvement) 

3.  Pass  a work-related  test  (what  type  of  test?) 

4.  Study  for  a GED 

5.  Other: 

* Can  yon  think  of  any  other  goals  you  have  that  aren’t  on  this  list? 


Of  all  the  goals  we’ve  talked  about,  what  are  2 or  3 that  are  the  most  important  to  you 
right  now? 

1. 

2. 

3. 
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Burke  Reading  Interview  (Modified) 


1.  When  you  are  reading  and  you  come  to  something  you  don’t  know,  what  do  you  do? 

Do  you  ever  do  anything  else? 

2.  Who  is  a good  reader  you  know? 

3.  What  makes a good  reader? 

4.  Do  you  think ever  comes  to  something  he/she  doesn’t  know? 

5.  What  do  you  think  he/she  does  about  it? 

(If  answer  is  NO:  Suppose comes  to  something  he/she  doesn’t  know.  What  do  you  think  he/ 

she  would  do?) 


6.  Do  you  think  that reads  everything  the  same  way?  For  example,  would  he/ she 

read  a want  ad  in  the  newspaper  the  same  way  as  a mystery  story? 


7.  If  you  know  that  someone  is  having  difficulty  reading  something,  how  would  you  helpthat  person? 


8 .  What  would  a teacher  do  to  help  that  person? 


9. 


What  would  you  like  to  do  better  as  a reader? 
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Looking  at  Your  Own  Reading  Behavior 


1.  Do  you  understand  most  of  what  you  read  in  class  in  reading  groups? 


2.  Do  you  participate  in  discussions  and  ask  questions  about  what  you  read  in  class  if  you  don’t  under- 
stand? 


3.  Do  you  try  to  predict  what’s  coming  when  you  read  by  looking  at  the  title,  pictures,  or  thinking  about 
what  you  already  know  about  the  subject  you’re  reading  about? 


4.  Do  you  ever  figure  out  a word  you’re  not  sure  of  by  reading  the  words  around  it  and  guessing  what 
word  would  make  sense? 


5 .  When  you  read  something,  do  you  ever  think  about  how  it  connects  to  your  own  life  or  to  things  you 
know? 


6.  Do  you  ask  yourself  questions  about  what  you  read? 


7.  Do  you  read  outside  of  class?  (If  so,  what  kinds  of  things,  how  often  and  when?) 


8.  What  are  your  biggest  problems  as  a reader? 


9.  What  are  your  strengths  when  it  comes  to  reading? 
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Reading  Progress  Checklist 


Reader: 


Teacher: 


Date: 


1.  Seems  to  understand  in-class  readings  1. 

2.  Reports  understanding  reading  outside  2. 

of  class 


3.  Participates  in  pre-reading  & 3. 

post-reading  discussions 

4.  Connects  reading  with  own  experience/  4. 

knowledge  and  questions  what  is  read 

5.  Participates  in  group  activities  designed  5. 

to  develop  reading  strategies 

6.  Understands  that  gaining  meaning  is  the  6. 

purpose  of  reading 

7.  Uses  prediction  to  make  sense  of  text  7. 

8 . Willing  to  guess  at  unfamiliar  words  8. 

9.  Uses  context  to  guess  at  unfamiliar  words  9. 

& make  sense  of  text 


10.  Uses  prior  knowledge  of  subject  to  make  10. 

sense  of  text 


11.  Changes  approach  to  reading  depending  11. 

on  the  type  of  text 

12.  Uses  graphophonic  cues  appropriately  12. 


Comments: 
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Writing  Progress  Checklist 


Name  Date 


Can  Needs  Not  N/A 

Do Work  Evident 


A.  Process: 

1.  Generates  language 
experience  stories 

2.  Able  to  select  topics 

to  write  about  2. 

3.  Willing  to  try  to  put 

words  on  paper  3, 

4.  Develops  a topic  with 

details  H. 

5.  Able  to  use  writing  for 

a variety  of  purposes  S. 

6.  Willing  to  revise 

O' 

7.  Using  feedback  from 

others  to  revise  j 

8.  Writes  entries  in  dialogue 

journal  . 

0* 

9.  Writes  entries  in  learning  log 

?. 

10.  Willing  to  share  some 

pieces  with  others  ^ 

% 

11.  Gaining  confidence  in  self 
as  a writer  //t 

Comments/Observations: 


O 

ERIC 
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Can 

Do 


Needs 

Work 


Evident  ^/A 


O 

ERLC 


B.  Handwriting: 

1.  Prints  upper  case 

lower  case  / 

2.  Leaves  spaces  between  words 

3.  Optional  Competency:  writes  cursive 

0 

C Spelling: 

1.  Willing  to  use  invented  spelling  / 

2.  Uses  personal  word  dictionary  to  improve 

spelling  and  record  sight  words  ^ 

3.  Uses  word  lists/dictionary/ thesaurus  to 

check  spelling  during  editing  phase  ^ 

4.  Writes  letter(s)  that  represent 

consonant  sounds  q 

if.  Writes  letters  that  represent  digraphs 
and  blends  s. 

6.  Writes  letter(s)  that  represent  long 
vowel  sounds: 

a.  Uses  a e,  ay,  ai,  to  make  long 
a sound” 

> 

i 

t 

b.  Uses  ee,  ea,  to  make  long  e sound  fo 

c.  Uses  i_e,  igh,  y,  to  make  long  i g. 

d.  Uses  o_e,  oa,  oe,  ow,  to  make  , 

long  o sound  d. 

e.  Uses  u_e,  ue,  to  make  long  n 

% Writes  letters  to  represent  short 
vowel  sounds  f. 

2.  Applies  the  following  spelling  rules: 
a.  -ff,  -11,  -ss,  -zz  fa 

b.  -k  or  -ck,  -ch  or  tch,  -ge  or  dge  / 

; . : i 
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Caa  Needs  Not  N/A 

Do Work  Evident  


% Uses  "to,  too,  and  two"  correctly  ^ 

to  Uses  "there,  their,' they’re"  correctly  ^ 

) 

\ 

//.  Other  homonyms  used  correctly 
(list  them): 

//. 

KL  Note  any  spelling  error  patterns 
observed: 

a 

- 

D.  Mechanics: 

1.  Proofreads  first  drafts  /. 

2.  Writes  in  complete  sentences 

3.  Uses  end  punctuation  correctly 

\3 

4.  Uses  editing  symbols  to  revise  and 

edit  personal  writing  ^ 

5.  Uses  capital  letters  to: 

a.  Begin  a sentence  Sg. 

b.  Name  people,  places  and  special  things  £ 

c.  Capitalize  the  major  words  in  a title  ^ 

6.  Uses  commas  to  separate  items  in  a list  . 

(0. 

7.  Uses  " " around  the  words  people  sav 

out  loud  7 

8.  Goes  down  to  a new  line  and  indents  when 
beginning  a new  paragraph  £ 

9.  Uses  an  apostrophe  to  show  possession  ^ 

10.  Uses  an  apostrophe  to  make  a contraction 

to. 

11.  Uses  verbs  in  the  correct  tense 
H. 

12.  Note  any  other  skills  writer 
has  developed: 
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© 1977  0 /Sherman  & J.  DeLawtr 

MISCUE  ANALYSIS  FORM 

Student Selection 

Date — Class Evaluator 

I.  DIALECT  EVIDENCE 


Prep 

PostU 


O 

ERIC 


II.  SOUND-LETTER  RELATIONSHIPS:  HIOT  SOME  NON 


HI<?H  SOME  NONE 

HIGH 

. & SOME 

punzzi 


III.  GRAMMATICAL  RELATIONSHIPS: 


ACCEPTABLE  UNACCEPTABLE  _ 

^*"  ■ V % 


Grammatical 

acceptability 


t 2 i * 5 > 7 » t io  ii  ii  ii  14  i;  it  i;  n n a « m a M a 


0~T 


ACCEPT. 


IV.  MEANING  RELATIONSHIPS: 


12345*7*9 

10  It  12  13  14  IS  lb  17  18  19  20  21  22  23  24  2S  YES  SOME  NO  % YES 

. • l i I 1 1 " I | | ] 

lorrccuon|  

1 

L_l_i 

1 

LU 

1 

I 1 

i i i 

tllll 

1 1 1 1 i I t i t : 

Mill! > L 1 

1 2 3 4 s 6 7 K a 10  It  Ii  13  14  15  16  17  18  19  JO  21  22 

23  24  25  YES  SOME  NO  % NO 

meaning 

change 

J 

Cl 

1 3 m 

V.  PATTERNS  OF  STRENGTH  & WEAKNESS: 


TOTAL  TOTAL 


Strength 
Some  strength 
Weakness 


* 

* 

it 

9 

o 

*9 

IV 

1 1 

1 z 

1 .» 

15 

in 

i / 

lO 

4U 

41 

44 

ZJ 

44 

49 

70 

i 

i 

1 

i i 

VI.  RETELLING: 


Character  identification 

of  IS 

Character  description 

of  10 

Events  in  succession 

of  35 

Summary  of  plot 

of  10 

Theme,  moral,  or  message 

of  20 

Intro,  of  pers.  exper. 

of  10 

Total  retelling  score 

of  100 

VIKA).  REPEATED  MISCUES: 


(B) .  NON-WORD  SUBSTITUTIONS. 

(C) .  OMISSIONS 

CAUSING  MEANING  CHANGE 


VIII.  BAR  GRAPH  SUMMARY: 


Retelling  score 
Strength 
Some  strength 
Weakness 

Meaning  change 

Correction 

•»  • 

Gram,  acceptability 
Graphic  similarity 

6? 


o to  jo  jo  40  5o  to  to  an  <n  im 


IX.  EVALUATION  & RECOMMENDATION 


BEST  COPY  AVAILABLE 
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LEARNING  CONTRACT 


What  do  you  want  to  do? 


How  do  you  plan  to  do  it? 


Reading  Goals: 


Writing  Goals: 


Other; 


Learner’s  Signature 

Teacher’s  Signature 

Date 

Contract  Review  Date: 

Learner’s  Comments: 

Teacher’s  Comments: 

3 

ERIC 
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GOALS  LIST 


Name: 


Date: 


Interviewer: 


Part  I:  In  your  own  words,  can  you  tell  me 


your  reasons  for  coming  to  school  now? 


Part  II:  Here  are  some  goals  other  students  In  this  oroanm 


mentioned.  Tell  me  if  this  is 
or  something  you  really  have 


Personal 

Read  write  your  name  and  address 
Read  signs  (which  ones): 

Read  labels/instructions 

Read/write  notes  to/from  family 

Read  and  write  shopping  lists 

Read  a calendar,  bus  schedules,  TV  guides 

Use  a phone  book 

Read  menus  or  recipes 

Read  bills 

Write  checks 

Read  maps 

Read  information  related  to  health 
Fill  out  forms 

Read/write  personal  letters 

Read  the  newspaper  (which  sections): 

Read  magazines  (which  ones) 

Use  a dictionary 
Improve  handwriting 


Children 

Read  to  your  children/grandchildren 
Ages: 

Help  children  with  homework 
Read/write  notes  from  school 
Take  part  in  school-related  meetings  and  events 

Personal  - Books  and  Writing 

Read  books  for  enjoyment  (what  kind  - adventure,  mystery,  romance,  historical,  books  about 
people): 


Read  books  to  get  information  (what  kind  - personal  research,  current  events,  jobs,  children, 
health,  religious,  hobbies,  entertainment): 


Write  for  yourself  (what  kinds  - journal  or  diary,  experiences  you’ve  had,  advice  for  others,  your 
opinions,  reports  about  something  you've  read,  your  life  story  or  autobiography,  other  stories, 
poems,  words  to  songs): 


Work 

Fill  out  a job  application 

Use  reading  to  find  out  about  jobs 

Use  reading  to  team  to  do  your  job  better  or  open  a business 

Read  and  write  notes  from  and  to  co-workers 

Read  or  write  work  reports,  logs,  announcements 

Fill  out  order  forms/lists 

Participate  in  work-related  meetings;  take  notes. 


Community 


Register  to  vote 
Apply  for  citizenship 
Read  leases/contracts 
Apply  for  a library  card 
Take  the  driving  test 

Participate  in  community  meetings/clubs/religious  meetings 
Join  a group  to  work  on  a problem 
Publish  a newsletter  or  other  writing 

Education 

Attend  a job  training  program 

Attend  classes  to  learn  something  new  (hobbies,  self-improvement) 

« Pass  a wOfk-rciaied  test 
■ Get  a GED 

Part  ill:  Can  you  think  of  any  other  goals  you  have  which  we  have  not  mentioned? 


Part  IV:  Of  ail  the  goals  we  have  mentioned,  name  two  or  three  which  are  most  important  to  you  right 
now. 


MANY  LITERACIES: 

Modules  for  Training  Adult  Beginning  Readers 
and  Tutors 

By  Marilyn  Gillespie 

Center  for  International  Education 
285  Hills  House  South 
University  of  Massachusetts 
Amherst,  MA  01003  USA 
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EDUCATIONAL  GOALS 
THESE  ARE  MY  EDUCATIONAL  GOALS. 
I WILL  WORK  ON  THEM  EACH  DAY. 


GOALS  LIST 

1. 


DATE  GOAL  ACHIEVED 


2. 


3. 


4. 


5. 


other  information  about  Goals : 
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NAME: 's  GOALS  LJST  for  EDUCATION  PLAN 


LIFE  SKILLS 
To  read  ... 

I know/ 
do  t h i s 

I 

li 

kno 

now 

would 
ke  to 
w more 
later 

Does  t 
of  any 
you? 

YES 

his  subject  remind  you 
thing  else  that  interests 

and  write  my  name 

and  write  my  address 

and  write  my  phone  number 

and  write  my  social  security 

la bels/ins t rue tions /cos t of... 

food 

medicine 

clothes 

appliances 

To  read  ... 
a calendar 

a bus  schedule 

t . v . guide 

a phone  book  . 

menus /reci pes 

bills 

* 

maps 

news  papers 

magazines 

health  information 

To  read  • ■ * 
and  write  checks 

and  write  letters 

and  fill  out  forms 

and  use  a dictionary 

O 

ERIC 


LIFE  SKILLS 
To  read... 

I know/ 
do  this 

I 

li 

knc 

now 

would 
.ke  to 
>w  more 
la ter  # 

Does  tl 
you  of 
that  ii 
YES 

his  sub j ec tr emind 
anything  else 
iterests  you? 

Children 

Read  to  ch i ldr en / gr andchil dr en 

Help  my  child  with  homework 

Write  notes  to  school 

Read  notes  from  school 

Personal 
reading  f or  . . . 
en j o y ment 

adventure 

mystery 

romance 

historical 

people  stories 

other: 

reading  for  * . . 
self-help 

recovery(alcohol/drugs) 

personal  interest 

current  events 

jobs 

children 

health 

physical 

emotional 

religious/ spiritual 

reading  f or  . * . 
hobbies 

entertailnent 

Personal  writing 
poems  /songs 

improve  handwriting 

To  write. . . 
j ournal/ diary 

advice  to  others 

work/school  reports 

newsletter 

to/for  newspaper 

Other:  Any  other  skills  you  would  be  interested  in  learning. 
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READING  SKILLS 

I know/ 
do  this 

I 

1: 

kn< 

now 

would 
Lke  to 
3w  more 
later 

Does 
you  o 

inter 

YES 

this  subjectremind 
f anythingelse  tha 
e s t s y o r;  : 

For  Wo r k : 

fill  out  a job  application 

read  want  ads 

read  notices 

read  reports 

fill  out  forms 

take  notes  at  meetings 

For  Community  Life: 
open  a checking  account 

register  to  vote 

apply  for  a library  card 

take  a learner's  permit  test 

For  English  and  other  skills: 
ca  pi t a 1 i zat ion 

punctuation 

spelling 

parts  of  speech(nouns , verbs) 

grammar / sent ence  structure 

definitions 

understanding  what  I read 

For  GED  Tests  : See  separate  sheets  for  specific  topic  breakdown. 

GED  Writing/Reading 


GED  Literature 

GED  Social  Studies 

GED  Science 

For  College  Reading; 
comprehension 

test  taking  skills 

term  papers 

O 

ERLC 


i ( 


LIFE  SKILLS 
for  math... 

I know/ 
do  this 

I w 
lik 
know 
now 

ould 
e to 
more 
later 

Does  th: 
you  of  a 
interest 
YES 

is  subject  remind 
.ny thing  else  that 
s you? 

Daily  life 

write  dates  as  numbers 

tell  time 

write  the  time 

find  a batting  average 

add/subtract  dollars /cents 

add/subtract  change 

estimate  shopping  $ total 

find  savings  from  sales /coupons 

use  unit  price  buying  food 

compare  cost  of  generic  and 
brand  name  medicine 

read  correct  drug  dosage 

read  thermometer 

use  cooking  measures 

find  total  calories 

know  nutrition  information 

measure  with  standard  ruler/tape 

measure  with  metric  ruler/tape 

convert  measures 

use  map... local 

state 

u.s. 

i 

find  car  mileage 

total  a restuarant  check 

figure  sales  tax 

read  sales  tax  chart 

\> 


LIFE  SKILLS 
for  math 

I know/ 
do  this 

i 

Li 

knc 

now 

would 
.ke  to 
►v  more 
later 

Does  t 
you  of 
inter e 

his  subject  remind 
anything  else  that 

sts  you? 

Budget:  rent,  food, clothes 

medical, education, fun, . . 

make  a budget 

set  savingsgoals 

find  annual  car  expense 

figure  total  cost  of  bill 

(phone, electric, heat , etc  . ) 
how  to  save  on  energy 

figure  credit  card  charges 

figure  total  cost  of 

installment  plan 

find  interest  on  money 

find  lowest  rental  cost 

use  a checkbook / pass  book 

i . 

deposit 

withdraw 

balance 

t 

Home  Improvement 

estimate  cost  of  home  repair 

find  dimensions  for 
scale  drawing 

figure  amount  of  materials 

(paint, insulation ,rug) 

Work 

take  a pre-employment  test 

understand  paycheck 

(net, gross , over-time) 
compare  benefits 

do  a supply  order 

schedule  work  hours 

Other 

any  other  skills  you  would  be  interested  in  learning 


ERIC 
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MATH  SKILLS 

I know/ 
do  this 

I w 
lik 
know 
now 

ould 
e to 
more 
later 

Does  thj 

you  of  £ 

interest 

YES 

ls  subject  remind 
inything  elsethat 
:s  you? 

Math  and  other  skills: 
addition 

subtraction 

multiplication 

division 

weights/measures 

measurement -standard 

measurement-metric 

telling  time 

decimals 

fractions 

ratio/proportion 

percent 

word  problems 

GED  Math:  See  seperate  sheets 
GED  Math 

for  specif 

ic  topi 

c breat 

cdo wn . 

Algebra 

Geome  try 

Work 

job  training  math 

* 

take  job  related  math  test 

College  Math 
test  taking  skills 

review  work 

o 

ERIC 
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MONTHLY  REVIEW 

WHAT  I LEARNED  IN  THE  MONTH  OF .19 


WHAT  I LEARNED  IN  THE  MONTH  OF ,19 


WHAT  I LEARNED  IN  THE  MONTH  OF ,19 


Remember : Think  of  what  you  have  learned  in  and  out  of  class. 


O 


EKLC 

hfliflaffHHaaaa 


GED 
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PRIORITY  MATH 


GEOMETRY 


Angles  Identify  a 

right  angle  in  a concrete  way 
in  a graphic  way 
in  a verbal  way 
and  as  a formula 

straight  angle  in  a concrete  way 
in  a graphic  way 
in  a verbal  way 
and  as  a formula 

a circle  in  a concrete  way 
in  a graphic  way 
in  a verbal  way 
and  as  a formula 

Draw  different  angles ( 45° , 90° , 180° ) 
and  estimate  angle  size 

Triangles 


CHECKLIST 

Check  here 
when 
done „ 


Check  Here  and  write 
book  name,  if  done  ±f) 
a different  book. 


Make  a straight  line  angle  into  a triangle 

Identify  isoseles  triangles 

equilateral  triangle 
right  triangles 

Compare  angles  in  adjacent  triangles 

Using  ratio/proportion, solve  problems 

triangles  and  Pathagorean  Formula 
Identify  right  triangles 

Read  and  practice  Pythagorean  Formula 

Solve  problems  using  Formula  C^»  .A^  + 

Coordinates 

Identify  a point  (ordered  pair) 

Plot  a point , given  X,Y 

Estimate  X,Y  coordinates , when  given  an 
ordered  pair 


ERIC 


GED 


PRIORITY  MATH  CHECKLIST 


Check  Here  and  write 
book  nametif  done  in 
a different  book. 


Check  Here 
when 
done  • 

ALGEBRA 

identify” set-up"  or  "the  expression  that 
represents"  problems  

solving  "set-up"  problems  

practice  "set-up"problems  

evaluate  expressions  and  formulas 

+ - 

x - 

parentheses  

with  exponents  • 

using  a formula  

equations 

guess  missing  numbers-verbal  

( 

guess  missing  numbers-written  

check  solution  

guess  missing  number-written  

strategies  for  doing  equations  


FRACTIONS  DECIMALS  PERCENTS 


identify  a fraction 

graphically 

verbally. 

as  a notation 

do  a fraction  activity 

physical 

verbal  practice 
ruler  fractions 


use  decimals  in  everyday  activities 
decimal  practice  worksheets 


use  percents  in  everyday  activities 

read  meaning  of  percent 

change  percent  to  decimal^—— 
change  percent  to  fraction 

change  decimal  to  percent  

memorize  basic  percent  meanings 

practice  problems  with  percent  grid 


G ED 


PRIORITY  MATH  CHECKLIST 


Check  Here  and  write 
book  name , if  done  in 
a different  book. 


PERIMETER, AREA  AND  VOLUME 
PERIMETER 

Identify  it  in  a concrete  way 
Identify  it  in  a graphic  way 
Identify  it  in  a verbal  way 
Identify  it  as  a formula/notation 
Do  a practice  sheet  using  a formula 
Find  perimeter  of  figures 
Setting  up  the  problem  worksheet 
Perimeter : Circumference  of  circle 
Do  Worksheets  for  practice 
AREA 

Identify  it  in  a concrete  way 
Identify  it  in  a graphic  way 
Identify  it  in  a verbal  way 
Identify  it  as  a formula/notation 
Find  the  area  worksheet 


VOLUME 

Identify  it  in  a concrete  way 
Identify  it  in  a graphic  way 
Identify  it  in  a verbal  way 
Identify  it  as  a formula/notation 
Find  volume  worksheet 
Finding  a value  worksheet 


FORMULA  PAGE 

Familiarity  with  the  Formula  Page 


Check  Here 
when 
done  • 


GED 


PRIORITY  MATH  CHECKLIST 


Check  Here  Check  Here  and  write 

when  book  name, if  done  in 

done  a different  book. 

NOTATION 

identify  different  ways  to  show  + - X 4*  

Rules  of  OrderrMy  Dear  Aunt  Sally  

exponents  

Excuse  My  Dear  Aunt  Sally  

exponents-squared  and  cubed  

scientific  notation  

radical  sign  

parentheses  

Please  Excuse  My  Dear  Aunt  Sally  

doing  PEMDAS  expressions  

reading  and  understanding  PEMDAS  

PROBLEM  SOLVING  

deciding  if  a problem  uses  + - X t 

identifying  words  used  in  + - X t 

practice  deciding  + or  - 

practice  deciding  I or  : 

sort  word  problems  by  + - X f 

LEARNING  STRATEGIES  FOR  SOLVING  PROBLEMS  ) 

read  and  discuss  different  solutions  

look  at  samples  of  solutions  

do  a review  

do"Solving  Word  Problem”packet 
check  answers 
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Foreword 


Loren  McGrail  

lllllilll  his  second  issue  of  Adventures  in  Assess- 

■M  merit  looks  at  ongoing  assessment-the 
ways  we  document  learning  as  it  takes 
place  and  the- ways  we  integrate  it  into  instruction. 
As  the  authors  in  this  volume  remind  us,  ongoing 
assessment  is  part  of  the  learning  process- “in- 
separable from  good  teaching  practice”  (Isserlis, 
p.  41).  Ongoing  assessment  is  essential  for  both 
learners  and  teachers  because  “it  helps  keep  learn- 
ers aware  of  their  own  progress  and  development  as 
learners.  It  keeps  teachers  aware  of  learners’  needs, 
which  is  vital  to  planning  relevant  and  effective 
teaching  activities”  ( Kelly,  p.  5). 

Ongoing  assessment-means  looking  at  learners’ 
progress  over  time  in  various  areas  which  may 
include  what  counts  as  changes  in: 

•reading 

•writing 

•oral  language  use 
• uses  of  literacy 

• social  changes  inside  and  outside 
the  classroom 

• personal  or  affective  abilities 
• metacognitive  strategies 
• mathematics 


Alternative  approaches  to  assessment  introduce 
procedures  that  enable  adult  literacy  students  to 
evaluate  their  own  experience  and  progress.  They 
also  help  teachers  to  “identify  the  strategies  stu- 
dents use  and  how  these  strategies  change  as  learn- 
ers progress”  (Robishaw,  p.  59).  Thus,  they  help 
to  inform  the  development  of  curriculum  which  is 
based  on  learners  evolving  progress  or  change  in  all 
or  some  of  the  above  areas. 

The  tools  and  procedures  that  have  been  devel- 
oped to  measure  these  changes  in  ongoing  assess- 
ment include: 

• journals  (personal  and'group) 

• writing  portfolios 

• checklists  and  charts  (process  and  progress) 

• class  accomplishments 

• teacher  logs 

• student  logs 

In  this  volume  of  Adventures  in  Assessment, 
seven  authors  share  their  thoughts,  experiments, 
and  questions  about  ongoing  assessment  with  the 
hope  that  other  ABE  practitioners  will  be  encour- 
aged to  adapt  these  tools  in  ways  that  make  sense 
for  their  own  teaching/leaming  contexts. 
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Defining  the  Portfolio 

It  is  important  to  note  that  there  are  at  least  three 
different  meanings  of  the  word  “portfolio”  in  use 
in  alternative  assessment  and  in  this  publication . A 
writing  portfolio  refers  to  collecting  student  writ- 
ing in  a folder  for  the  purpose  of  recording  and 
documenting  growth  over  a period  of  time.  The 
progress  portfolio  is  a collection  of  all  the  proce- 
dures and  tools  a teacher  might  use  for  assessing 
progress  or  change  (i.e.,  the  toolkit).  Both  Paul 
Trunnell  and  Janet  Kelly  use  “progress  portfolio” 
to  refer  to  documenting,  collecting,  and  analyzing 
various  kinds  of  assessors  which  include  everything 
from  standardized  tests  to  writing  process  check- 
lists. Finally,  the  presentation  portfolio  (as  de- 
scribed by  David  Rosen)  shares  many  of  the  same 
qualities  as  the  progress  portfolio  but  is  also  used 
by  the  learner  for  presentation  purposes  much  like 
an  artist’s  portfolio. 

The  Toolmaker  and  the  Tool 

Portfolio  approaches  to  assessment  are  the 
current  rage  in  K -12  education  and  now,  increas- 
ingly in  adult  basic  education.  The  danger  in  both 
contexts  is  that  we  may  recreate  some  of  the 
problems  of  the  very  system  we  are  trying  to 
change.  If  all  we  do  is  substitute  new  multiple 
measures  for  old  standardized  measurements  and 
monitor  student  progress  for  diagnostic  purposes  in 
terms  of  identifying  strengths  and  weaknesses  in 
language  and  content  areas,  we  will  not  have 
created  a new  paradigm.  If  on  the  other  hand  we 
include  learners  as  active  participants  at  the  center 
of  the  process  of  measurement,  as  “co-investiga- 
tors in  determining  their  own  literacy  practices, 
strengths,  and  strategies,”  (Lytle,  1988)  then  we 
have  truly  engaged  in  alternative  assessment.  In 
addition,  if  we  only  look  for  change  in  language  and 
content  areas,  we  miss  the  chance  to  document 


change  in  other  areas,  for  example,  affective  changes 
such  as  improved  self-confidence,  the  ability  to 
participate  in  a group,  or  the  ability  to  use  literacy 
to  address  changes  in  family  or  community  life. 

Finally,  alternative  assessment  must  serve  the 
development  of  a curriculum  process  and  of  adult 
literacy  programs  as  a whole.  We  need  to  be  clear 
that  the  pedagogical  beliefs  and  assumptions  that  we 
espouse  are  truly  reflected  in  the  procedures  and 
tools  we  are  developing.  First  came  the  toolmaker, 
then  the  tool. 
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Introduction 


“We  must  be  less  technical  and  more  criti- 
cally reflective  of  what  we  do  and  are,  that 
we  must  set  aside  our  ‘How  ? ’ questions  and 
begin  to  ask  and  reask  ‘What?’ and  ‘Why?’” 

Virginia  Sauve 

The  contributors  to  Volume  Two  of  Adventures 
in  Assessment  are  practitioners  who  have  written 
critical  reflections  about  what  and  why  as  well  as 
how  they  have  developed  ongoing  assessment  pro- 
cedures and  tools. 

“The  Progress  Portfolio”  by  David  Rosen 
describes  an  assessment-presentation  model  cur- 
rently being  tried  with  young  adults  throughout  the 
country.  As  Rosen  states,  “this  portfolio  model 
has  the  advantage  of  providing  direct,  perfor- 
mance-based assessment  of  writing,  mathematics, 
problem-solving,  oral  presentation*  job  readiness, 
and  even  vocational  skills.  ” 

In  “Keeping  Us  Aware,”  Janet  Kelly  contin- 
ues her  description  of  the  procedures,  tools,  and 
processes  that  make  up  her  portfolio  approach  in  a 
“learning  community.  ” As  in  Volume  One,  she 
has  included  the  tools  and  forms  with  a focus  this 
time  on  reading  and  writing  assessment  and  learn- 
ing logs  kept  by  both  learners  and  teachers. 


Paul  Trunnell,  from  Harborside  Community 
Center  in  East  Boston,  describes  in  “Self-Assess- 
ment: Doing  and  Reflecting”  a portfolio  approach 
similar  to  Janet  Kelly  ’ s for  assessment  with  his  ABE 
2 students.  He  describes,  in  detail,  two  ongoing 
assessment  procedures  he  has  had  success  with:  the 
learning  log  and  the  skills  evaluation  sheet. 

In  “When  Asking  Isn’t  Enough,”  Kathy 
Brucker  documents  a variety  of  evaluation  tools 
she  has  used  with  her  Spanish  speaking  students  at 
El  Centro  del  Cardenal  in  Boston.  Her  critique  of 
checklists  that  don’t  ask  “which  did  you  enjoy  and 
why?”  reminds  us  all  that  we  need  to  adapt  tools  to 
fit  our  own  contexts  and  answer  our  own  questions. 

Janet  Isserlis  from  the  International  Institute  in 
Providence,  Rhode  Island  writes  in  “What  You 
See:  Ongoing  Assessment  in  the  ESL/Literacy 
Classroom”  about  action  research  on  how  adult 
learners  progress  in  and  out  of  the  classroom.  Her 
focus  is  on  ongoing  observation,  reflection,  and 
feedback  aimed  at  assisting  learners  and  facilita- 
tors. Her  anecdotal  reports  provide  a model  for 
what,  why,  and  how  to  write  up  daily  observations. 

Karen  Ebbit,  Priscilla  Lee,  Pam  Nelson,  and 
Joann  Wheeler  from  the  Community  Learning 
Center  in  Cambridge  describe  in  “Three  by  Three 
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by  Four:  Ongoing  Assessment  at  the  Community 
Learning  Center”  their  cooperative  effort  to  de- 
velop assessment  tools  used  with  ESL/literacy 
students.  They  describe  the  creation  of  a “three- 
part/three-tool”  process  which  includes  an  initial 
goal-setting  exercise,  a weekly  self-assessment, 
and  a progress  record.  The  tools  were  field  tested 
in  three  ESL  classes  during  the  summer  of  1991. 

In  “Further  Adventures  in  Alternative  Assess- 
ment: an  Annotated  Bibliography,”  Don  Robishaw 
outlines  some  key  principles  underlying  ongoing 
assessment  and  cites  four  annotations  from  the 
bibliography. 

While  all  of  these  writings  fall  under  ongoing 
assessment,  there  are  many  other  lenses  through 
which  to  view  and  analyze  these  writings.  One  way 
to  look  at  these  articles  is  to  examine  the  myriad 
ways  practitioners  have  gone  about  documenting 
their  experiments.  For  example,  even  though  all 
three  teachers  at  the  Community  Learning  Center 
were  testing  a jointly  designed  initial  goal-setting 
tool,  each  of  them  describes  different  objectives  for 
doing  this  activity.  Compare  the  evolution  of  the 
forms  and  tools  Paul  Trunnel  developed  for  his 
ABE  learners  to  the  way.Brucker  developed  her 
tools.  How  are  their  processes  similar?  How  are 
they  different?  What  kinds  of  information  do  these 
tools  give?  How  do  they  compare  to  the  kind  of 
anecdotal  reporting  Isserlis  does  daily  in  her  class- 
room? If  progress  is  achieved,  for  whom  is  it 
achieved  and  by  whom? 

Another  way  to  look  at  these  writings  is  to  try 
to  determine  to  what  extent  they  document  the  kinds 
of  changes  mentioned  in  the  foreword  and  how  they 
accomplish  this.  Which  tools  work  best  for  measur- 
ing which  kinds  of  change  or  progress?  Which  ones 
work  best  with  which  kinds  of  learners? 
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It  is  my  personal  hope  that  future  editions  of  this 
journal  will  include  reactions  or  comments  to  these 
writings  in  the  form  of  letters  to  the  editor  or 
descriptions  of  a tool  or  procedure  borrowed  and 
tested  out.  I would  also  like  to  encourage  people  to 
write  about  the  process  involved  in  choosing  and 
developing  certain  tools-the  burning  issues  and 
questions  that  prompted  you  to  explore  another 
form  of  assessment.  Although  the  backbone  of  this 
publication  is  the  research  and  writing  of  Massachu- 
setts classroom  teachers,  in  future  issues  I will 
continue  to  encourage  submissions  from  practition- 
ers who  are  not  classroom  teachers,  such  as  David 
Rosen,  Interim  Director  of  the  Adult  Literacy 
Resource  Institute  in  Boston  or  from  out-of-state 
practitioners  such  as  Janet  Isserlis  from  Rhode 
Island  who  has  been  a pioneer  teacher  researcher  in 
adult  literacy.  I would  like  to  encourage  anyone  and 
everyone  to  think  about  reviewing  and  annotating 
an  article  or  book  for  this  journal  and  for  the 
annotated  assessment  notebook  at  your  SABES 
Regional  Support  Center. 

I would  like  to  express  my  thanks  to  the  SABES 
Central  Resource  Center  staff  who  contributed  their 
time  and  skill  to  this  project:  Laura  Purdom,  Sally 
Waldron,  and  Lou  Wolrab. 

Volume  Three  of  Adventures  in  Assessment  is 
expected  out  in  the  fall  of  1993  and  will  focus  on 
“Looking  Back”  activities  that  come  at  the  end  of 
a cycle  and  that  help  teachers  and  learners  reflect 
on  what  they  have  and  haven  ’ t accomplished . Many 
of  the  activities  in  “Looking  Back”  will  be  the 
same  as  in  earlier  volumes  and  will  look  at  progress 
over  a specific  period  of  time.  Thus,  we  will  be 
looking  for  writings  that  fit  into  all  three  compo- 
nents: initial,  ongoing  and  end-of-cycle  assessment 
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with  a special  emphasis  on  class  and  program 
evaluations. 

— Loren  McGrail 


Please  refer  to  p.  62  for  information  on 
how  to  submit  articles.  Deadline  for 
Volume  3 is  June  14, 1992. 
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The  Progress  Portfolio 


David  J.  Rosen 


ISilMince  the  end  of  the  1980’s,  writing  portfolio 
:rf  assessment  has  become  a hot  idea  in  public 
schools.  Teachers  are  using  writing  port- 
folios in  major  urban  school  systems  in  Pittsburgh 
and  Rochester.  Vermont  includes  them  in  its 
statewide  school  assessment  process.  And  since 
1990,  the  National  Assessment  of  Educational 
Progress  (NAEP)  has  added  portfolio  assessment  to 
its  list  of  ways  to  measure  student  writing . Portfolio 
assessment  has  spread  so  quickly  that  by  the  spring 
of  1990,  the  Northwest  Regional  Educational 
Laboratory’s  test  center  could  establish  a portfolio 
information  clearinghouse  to  publish  and  update  a 
bibliography  on  portfolio  assessment  practices. 

Not  surprisingly,  teachers  of  adults  and  out-of- 
school young  adults  have  also  been  experimenting 
with  portfolios.  As  earlyas  September  1989,  in  an 
article  which  appeared  in  the  New  York  City 
Literacy  Assistance  Center  Update,  LAC  staff 
member,  Sara  Hill  suggested  that  adult  literacy 
programs  could  use  writing  assessment  portfolios 
and  described  them  as  follows: 

“The  writing  folder  or  portfolio  is  a 
way  of  keeping  track  of  the  changes  in 
individual  student  writing  and  con- 
tains all  writing  from  the  beginning  of 
the  class-scraps,  notes,  drawings,  lists, 
drafts,  revisions,  final  pieces,  etc. 
Journal  entries,  too,  may  be  important 


to  keep  in  the  folder.  All  should  be 
dated  so  that  you  can  have  a clear  sense 
of  writing  growth  and  both  student  and 
teacher  should  have  access  to  it— per- 
haps keeping  it  in  a special  ‘folder 
box.’  From  time  to  time  the  teacher 
and  student  should  go  through  the 
folder,  with  the  student  selecting  ‘fa- 
vorite’ and  ‘least  favorite  pieces’  and 
talking  about  what  worked  and  didn’t 
work.  Teacher  and  student  might  also 
note  changes  in  the  spelling  and  me- 
chanics of  writing  over  time  and 
whether  or  not  a student  is  revising  or 
has  discovered  new  revision  strate- 
gies...” 

While  much  attention  has  been  given  to  whole 
language  portfolios  as  a means  of  assessing  writing, 
there  is  another  portfolio  model,  less-known  but 
more  comprehensive.  With  this  model,  learners 
and  teachers  can  assess  writing  and  other  basic  skills 
improvement.  And  with  this  model  a learner  can 
also  document  a wide  range  of  learning  accomplish- 
ments and  polish  and  present  them  to  others. 

The  Purpose 

The  model  is  a blend  of  writing  folder  assess- 
ment and  a tangible  collection  of  finished  products 
which  is  something  like  an  artist’s  portfolio.  This 
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assessment-presentation  model  is  currently  being 
tried  with  young  adults  in  a number  of  youth  service 
corps  around  the  country.  This  portfolio  model  has 
the  advantage  of  providing  direct,  performance- 
based  assessment  of  writing,  mathematics,  prob- 
lem-solving, oral  presentation,  job  readiness,  and 
even  vocational  skills.  It  uses  simple,  low-cost 
materials  such  as  a folder,  three-ring  binder,  and 
cardboard  artist’s  portfolio  case,  and  can  include 
written  and  audio-  and  video-recorded  demonstra- 
tions of  skill  and  knowledge. 

The  presentation  portfolio  has  several  pur- 
poses: 

• to  provide  a single  place  where  a 
learner  can  record  and  keep  evidence 
of  learning  from  other  parts  of  his  or 
her  life; 

• to  be  a way  a learner  can  show 
family,  friends,  teachers,  prospective 
employers  and  training  programs,  and 
college  admissions  officers  what  that 
person  can  do; 

• to  recognize  a learner’ s accomplish- 
ments; 

• to  be  a learner-centered  assessment 
process,  one  which  begins  with  the 
student’s  needs  and  goals  in  a student- 
created  learning  plan,  links  assessment 
and  documentation  of  achievement  to 
those  goals,  and  puts  choices  about 
what,  when,  and  how  assessment  is 
done  in  the  control  of  the  learner;  and 

• to  provide  an  organized  way  to  collect 
and  effectively  present,  direct  evidence 
of  learning  which  a learner  may  need 
for  “attainment  of  competencies,” 


certification,  graduation,  getting  a job, 
and  entering  training  and/or  college. 

As  with  a writing  assessment  portfolio,  the 
focus  of  this  model  is  on  learner  growth,  but  here 
attention  is  also  paid  to  recognizing  and  effectively 
presenting  achievement  to  others.  As  with  writing 
assessment  portfolio  models,  learners  keep  their 
writings  in  a folder,  and  they  are  periodically 
reviewed  for  growth;  but  in  this  model  learners  also 
polish  and  organize  writings  and  other  evidence  of 
learning  for  presentation.  Every  learner  has  a three- 
ring  loose-leaf  binder  with  tabbed  dividers  and 
plastic  sheet  protectors.  Some  learners  may  also 
have  a cardboard  artist’s  portfolio  case  to  hold 
nonprint  items  such  as  audio  and  video  cassettes,  or 
other,  larger  evidences  of  learning. 

In  the  binder,  or  achievement  portfolio,  learn- 
ers can  place  selected  samples  of  their  best  written 
work:  poems,  original  song  lyrics,  a short  autobi- 
ography, stories,  essays,  letters,  a resume,  a com- 
pleted college  application,  and  before-and-after 
photographs  of  visually-oriented  projects  they  have 
done  on  their  own  or  with  others.  Learners  can  also 
include  other  evidence  of  newly-acquired  abilities, 
such  as  results  of  written  tests  which  they  believe 
demonstrate  their  skills  or  knowledge,  or  letters 
others  have  written  about  their  knowledge,  skills, 
or  character. 

Although  the  achievement  portfolio,  sometimes 
called  a presentation  portfolio  or  interview  portfo- 
lio, might  include  some  materials  describing  the 
learning  center  or  program,  the  purpose  is  to  put  the 
learner’s  work  in  context,  not  to  document  the 
program’s  achievements.  Also  the  achievement 
portfolio  is  intended  to  show  the  learner’s  accom- 
plishments in  meeting  her  or  his  own  goals,  not  to 
compare  one  learner  with  one  another. 

Each  item  included  in  the  loose-leaf  binder  or 
portfolio  case  should  be  a clear  piece  of  evidence  of 
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the  learner’s  ability  to  do  something,  for  example: 

• to  show  clarity  about  one’s  personal 
and  career  goals  a learner  includes  a list 
of  goals  and  a short  plan  for  future 
learning  and  skill  development  with  her 
anticipated  timeframes; 

• to  demonstrate  his  knowledge  of  com- 
munity resources,  his  ability  to  do  re- 
search by  telephone,  and  to  present  the 
written  results  of  his  research  a learner 
includes  a five-page,  word-processed 
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childcare  resources  which  are  located 
near  the  learning  center  (he  also  did  the 
word  processing); 

• as  evidence  of  her  ability  to  measure 
area,  and  to  paint  walls,  a student  in- 
cludes before-and-after  photographs, 
including  close-ups,  of  the  apartment 
she  painted;  with  these  she  includes  the 
sheet  which  shows  how  she  calculated 
how  much  paint  she  needed; 

• to  show  his  understanding  of  how  state 
government  works,  a student  includes 
his  word  processed  letters  to  his  state 
senator  and  representative  urging  sup- 
port for  a bill  to  increase  funding  for 
adult  literacy  education; 

• to  show  his  readiness  for  work  as  a 
commercial  driver,  a learner  includes 
his  new  license.  (This  was  his  main 
goal  when  he  entered  the  center.); 

• to  demonstrate  group  presentation  and 
facilitation  skills,  a learner  includes  in 
her  portfolio  case  a videotape  of  her 


facilitating  discussion  in  a study  circle, 
and  of  her  presentation  to  the  learning 
center  advisory  committee  on  the 
center’s  new  learning  progress  assess- 
ment tool:  portfolios!; 

• as  evidence  of  readiness  to  enter  col- 
lege, a learner  includes  her  GED  dip- 
loma, a completed  Financial  Aid  form, 
and  completed  applications  to  three  col- 
leges; and 

• as  one  piece  of  evidence  of  work  readi- 
ness, a young  adult  learner  includes  the 
awnrd  he  received  for  perfect  class  sit- 
tendance. 

The  Process 

A student  begins  the  portfolio  process  by  creat- 
ing a development  (or  everyday)  portfolio  when 
she  enrolls  in  the  learning  center.  Initial  assess- 
ments of  needs,  learning  goals,  and  possibly  a 
learning  style  inventory  can  be  included  in  this 
catch-all  collection  which  is  kept  in  a large  expand- 
able folder  or  Pendaflex  folder  containing  separate 
file  folders  for  each  kind  of  work.  Each  week,  she 
adds  completed  pieces  and  work-in-process  to  the 
development  portfolio  and  she  reviews  it  informally 
with  her  teacher(s)  and/or  other  learners  to  assess 
her  progress  in  attaining  her  goals. 

At  the  end  of  each  cycle,  she  uses  the  develop- 
ment portfolio  for  a more  formal  review  of  her 
progress,  to  determine  areas  of  improvement,  and 
to  revise  old  goals  and  set  new  ones.  At  this  time 
she  may  also  decide  what  audiences  she  hopes  to 
present  the  achievement  portfolio  to  and  she  may 
select  (and  possibly  polish)  the  pieces  she  wishes  to 
put  in  her  achievement  portfolio  binder  or  case  for 
presentation.  In  programs  which  have  specific 
graduation  requirements,  such  as  adult,  external  or 
alternative  diploma  programs,  a learner  may  incor- 
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porate  these  goals  and  objectives  in  her  learning 
plan,  and  the  divisions  of  her  achievement  portfolio 
might  reflect  these.  Where  students  are  at  beginning 
levels  of  basic  skills,  or  English  language  learning, 
the  achievement  portfolio  will  most  likely  reflect 
intermediate  accomplishments,  and  the  audiences 
are  more  likely  to  be  the  learner  herself,  family, 
friends,  and  teachers  or  tutors.  More  advanced 
learners  will  probably  select  accomplishments  for  an 
audience  of  current  or  prospective  employers  or 
training  programs  or  college  admissions  personnel. 

As  the  achievement  portfolio  grows,  the  learner 
will  add  new  tabbed  dividers.  Depending  on  the 
learner’s  interests  and  the  type  of  learning  center 
program,  tabbed  sections  might  include:  personal 
learning  plan  (including  needs  and  goals);  compe- 
tency checklists;  test  results;  licenses,  certificates 
and  awards;  work  readiness  documents  (includ- 
ing a resume  and  any  employer  letters  of  recommen- 
dation); and  writing  samples  (typed  poems,  essays, 
letters,  etc.).  Some  learners  will  want  to  emphasize 
personal  creativity;  others  will  focus  on  work  readi- 
ness; many  will  want  to  stress  academic  achieve- 
ment. As  the  portfolio’ s purpose  is  to  represent  the 
learner’s  achievement  from  her  point  of  view,  these 
decisions  should  be  left  to  the  learner.  Teachers  and 
other  learners  should,  of  course,  feel  free  to  make 
suggestions. 

When  a learner  is  ready  to  use  the  achievement 
portfolio  for  employment  or  entering  further  educa- 
tion or  training , program  staff  or  other  learners  assist 
her  in  learning  strategies  for  presenting  it.  The 
achievement  portfolio  is  intended  to  be  a flexible 
tool.  A loose-leaf  binder  is  used  so  that  its  contents 
can  be  easily  reorganized  for  a specific  audience, 
including  only  those  accomplishments  which  are 
pertinent  to  that  audience. 

The  learner  should  not  rely  on  the  binder  or  audio 
or  videotapes  to  present  herself,  but  should  use  these 
as  props  for  presenting  herself.  She  should  also  have 


many  opportunities  to  role  play,  rehearse  or  prac- 
tice presenting  her  achievement  portfolio  materi- 
als with  other  learners,  learning  center  staff,  and 
visitors  as  her  audience. 

David  Rosen  is  Interim  Director  of  the  Adult 
Literacy  Resource  Institute  in  Boston. 


Notes 

“Northwest  Report,”  The  newsletter  of  the  Northwest 
Regional  Educational  Laboratory,  April,  1990.  Avail- 
able at  no  charge  from  NWREL,  101  S.W.  Main  St., 
Suite  500,  Portland,  OR  97204. 

Hill,  S.  “Alternative  Assessment  Strategies:  Some  Sugges- 
tions for  Teachers.”  Vol.  VI,  No.  1,  (September, 
1989)  issues  of  Update,  a publication  of  the  Literacy 
Assistance  Center,  15  Dutch  St.,  4th  Floor,  New 
York,  NY  10038. 

Information  about  a national  youth  service  corps  portfolio 
assessment  model  is  available  from  the  Urban  Corps 
Expansion  Project,  Public/Private  Ventures,  399  Mar- 
ket St.,  Philadelphia,  PA  19106 
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Keeping  Us  Aware 


Janet  Kelly 


illiy  ngoing  assessment  is  essential  for  learners 
lf||j|:V  and  teachers.  Effective  ongoing  assess- 
ment  helps  keep  learners  aware  of  then- 
own  progress  and  development.  It  also  keeps 
teachers  aware  of  learners’  needs,  which  is  vital  to 
planning  relevant  and  effective  activities . At  Read/ 
Write/Now,  we  have  been  using  a variety  of  tools 
as  part  of  ongoing  assessment.  Some  of  them  are 
used  primarily  by  teachers,  some  are  used  by 
learners,  and  some  are  shared.  As  much  as  pos- 
sible, the  tools  and  process  of  ongoing  assessment 
are  integrated  into  the  curriculum. 

The  key  element  that  gives  assessment  meaning 
is  consistent  communication  with  every  learner. 
This  is  formalized  in  conferences  and  with  the  help 
of  particular  forms,  but  the  communication  about 
learning  that  happens  amor^g Jeamers  and  between 
learners  and  teachers  is  largely  informal  and  hap- 
pens in  virtually  every  aspect  of  classroom  life. 
Classes  are  small  (usually  not  more  than  12  learners 
in  a class)  and  teachers  and  trained  tutors  are 
accessible  and  interested  in  what  learners  have  to 
say  about  their  lives,  the  things  they  care  about,  and 
what  they  think  about  their  own  learning. 

Looking  back  at  dialogue  journals  and  the  work 


in  a learner’s  writing  folder  can  be  used  to  see 
writing  development.  Showing  a learner,  who  is 
discouraged  about  spelling,  the  first  entry  s/he 
made  in  a dialogue  journal  or  the  first  draft  of  an 
earlier  piece  of  writing  and  comparing  it  with  the 
current  work  is  a tangible  and  very  convincing  way 
to  document  the  progress  s/he  has  made. 

Periodic  review  of  students’  learning  contracts 
to  look  at  progress  toward  meeting  goals  and  to 
revise  goals  or  plans  to  meet  them  are  also  part  of 
ongoing  assessment  in  our  program.  This  winter, 
we  began  the  class  cycle  with  a 20  minute  individual 
conference  with  each  learner.  Since  all  were 
returning  students,  we  used  the  conferences  to 
review  their  goals  with  them  and  revise  their 
learning  contracts  as  needed.  With  a few  learners 
whose  attendance  and  commitment  had  been  a 
problem  during  the  previous  cycle,  the  conferences 
were  an  opportunity  to  discuss  any  barriers  to  their 
learning  and  ways  to  overcome  the  barriers,  with  a 
clear  plan  of  things  we  could  do  and  things  they 
could  do.  This  seemed  to  work  well.  It  gave  both 
teachers  and  learners  an  opportunity  to  articulate 
what  our  individual  and  shared  responsibilities  are 
as  members  of  a learning  community. 


* See  Adventures  in  Assessment,  Vol.  1 "Read/Write/Now  Adult  Learning  Center  Assessment  Adventures”  for  a 
description  of  miscue  analysis. 
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The  Tools 

The  tools  and  organizers  we  use  as  part  of 
ongoing  assessment  are: 

• miscue  analysis* 

•booklist 

• reading  conference  record 

• reading  progress  checklist 

•teacher’s  log 

•learning  log 

• math  activities  record 

• writing  progress  checklist 

• writing  process  checklist 

• spelling  self-test 

• writing  conference  record 

• dialogue  journals 

This  may  seem  like  a lot  of  paper,  and  it  is.  At 
this  point,  all  of  these  forms  are  useful  to  our 
program.  We  are  continually  struggling  to  strike  an 
appropriate  balance  between  “wouldn’t  it  be  won- 
derful if  we  could  write  down  everything  that  shows 
what  people  are  learning  and  they  could  share  in  the 
process”  and  the  real  limitations  of  time.  What  is 
most  important  to  keep  asking  ourselves  is,  “Who 
and  whatis  this  for?”  If  a particular  tool  or  process 
makes  sense  in  the  light  of  that  question--if  it  serves 
the  purpose  of  telling  learners  and  teachers  more 
about  learners’  progress— it  stays  until  we  find 
something  better. 

The  Portfolio 

Read/Write/Now  uses  a portfolio  approach  to 
maintain  a comprehensive  record  of  ongoing  as- 
sessment. A learner’s  portfolio  is  added  to  and 
shared  with  the  learner  during  regular  conferences 
throughout  the  class  cycle  and  at  other  times  when 
a learner  needs  to  see  concrete  evidence  of  his/her 
progress.  The  portfolio  can  also  be  used  to  docu- 
ment progress  to  funders.  Although  this  approach 


tends  to  be  more  comprehensive  than  many  funders 
have  time  or  interest  in  seeing,  it  helps  to  organize 
evaluation  materials  in  a way  that  makes  writing 
narrative  summaries  for  funding  reports  easier. 

Sample  materials  used  at  Read/Write/Now  for 
ongoing  assessment  follow.  Descriptions  of  "what, 
why,  how,  and  when"  precede  each  form. 


Booklist 

What:  form  for  learners  to  keep  track  of 
books  they  read  in  and  out  of  class;  part  of 
assessment  portfolio 

Why:  to  show  learners  concrete  evidence  of 
accomplishment  in  reading ; to  help  teachers 
identify  interests  of  readers  to  help  make 
recommendations  for  other  books 

How:  learner  is  encouraged  to  keep  list  in 
folder  and  add  each  title  as  s/he  finishes 
book 

When:  ongoing 
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BOOK  LIST 


NAME: 

NAMES  OF  BOOKS  READ 

1. 

2. 

3.  

4.  

5.  

6.  

7.  

8.  

9. 


10. 

11. 

12. 

13. 

14. 
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DATE  FINISHED 


Reading  Progress  Checklist 

What:  teacher’s  short  checklist  for  assess- 
ment of  individual  learner' s reading  behavior 
and  development  over  period  of  class  cycle 

Why:  to  organize  and  categorize  reading 
assessment  information  in  an  accessible  for- 
mat; to  help  teachers  plan  strategy  lessons 
and  activities  for  individuals  and  reading 
groups 

How:  teacher  considers  learner's  reading 
behavior  in  class,  reading  conferences,  mis- 
cue  analysis,  learner's  self-assessment,  and 
responses  to  modified  Burke  Interview1" 

When:  after  learner  has  been  in  classes  for 
approximately  three  weeks  and  after  above 
activities  are  completed;  repeated  at  end  of 
cycle  to  assess  changes 


* See  Adventures  in  Assessment,  Vol.  1 "Read/Write/Now  Adult  Learning  Center  Assessment  Adventures"  for  a 
description  of  the  modified  Burke  interview. 
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Reading  Progress  Checklist 


Reader: 


Teacher: 


Date: 


1.  Seems  to  understand  in-class  readings 


2.  Reports  understanding  reading  outside 
of  class 

3.  Participates  in  pre-reading  & 
post-reading  discussions 

4.  Connects  reading  with  own  experience/ 
knowledge  and  questions  what  is  read 

5.  Participates  in  group  activities  designed 
to  develop  reading  strategies 

6.  Understands  that  gaining  meaning  is  the 
purpose  of  reading 

7.  Uses  prediction  to  make  sense  of  text 

8.  Willing  to  guess  at  unfamiliar  words 


9.  Uses  context  to  guess  at  unfamiliar  words 
& make  sense  of  text 

10.  Uses  prior  knowledge  of  subject  to  make 
sense  of  text 

11.  Changes  approach  to  reading  depending 
on  the  type  of  text 

12.  Uses  graphophonic  cues  appropriately 


Often  Sometimes  Rarely  Not  Yet 

Evident  Evident  Evident  Evident 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8 - 

9. 

10. 

11. 

12. 


Comments: 
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Reading  Conference  Record 

What:  a form  to  be  used  by  either  teachers 
or  readers  to  record  questions,  observations, 
and  reflections  about  sustained  reading 

Why:  for  readers  to  keep  a running  record 
of  words  they  want  to  find  out  about,  ques- 
tions they  have  about  what  they  are  reading, 
pages  read,  etc.  For  teachers  to  keep  a 
running  record  of  individual  reading  confer- 
ences, learners’  experience  of  reading  par- 
ticular texts,  problem  words,  observations 
about  strategies  used  during  brief  oral  read- 
ing samples,  responses  to  stories,  evidence 
of  comprehension  in  retelling 

How:  readers  or  teachers  use  each  page  to 
record  reading  responses  or  conference  re- 
sults for  two  different  class  days.  Reading 
conference  records  for  each  learner  are  kept 
-and -used  as  part  of-their  assessment  portfo- 
lios (See  also,  appendix  1,  "Suggestions  for 
Individual  Reading  Conferences. ") 

When:  whenever  individual  reading  confer- 
ences are  done;  usually  once  weekly  when 
individual  sustained  silent  reading  replaces 
reading  in  small  groups 


0 n 
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READING  CONFERENCE  RECORD 


Reader: Teacher: 


Date: Book: 


Start  Page 

finish  Page 

Date: 

Book: 

Start  Page 

finish  Page 
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Teacher's  Log:  Observations 
and  Reflections 

What:  a running  log  of  observations,  anec- 
dotes, and  critical  incidents  related  to  learn- 
ers' reading,  writing,  and  other  develop- 
ment. Each  page  has  room  for  brief  notes 
on  six  learners:  observations,  reflections, 
and  ideas  for  what  to  do  next  either  in 
response  to  the  particular  need  noted  or  in 
general. 

Why:  to  help  teachers  use  what  learners  do 
and  say  about  their  own  learning  to  direct 
curriuculm  planning,  as  well  as  to  keep  a 
record  of  learners'  stories  about  literacy 
growth  and  barriers  to  literacy  in  and  out  of 
class  to  help  us  respond  with  relevant 
activities 

How:  teachers  reflect  and  write  briefly 
about  an  agreed  upon  number  of  learners. 
We  use  our  reading  groups  since  it  works 
out  to  be  about  the  same  number  of  students 
in  each  group. 

When:  logs  are  done  as  soon  as  possible 
after  the  class  ends,  but  sometimes  that 
means  doing  the  logs  at  home  or  on  the  next 
day  before  planning  the  day's  classes 
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TEACHER’S  LOG:  OBSERVATIONS  & REFLECTIONS 


13 


Learning  Log 

What:  a log  or  journal  in  which  learners 
reflect  on  their  learning  for  the  week  by 
responding  to  four  questions 

Why:  to  provide  a format  for  learners  to 
think  about  their  learning  on  paper,  com- 
plain, criticize,  feel  good  about  accomplish- 
ments, recognize  progress,  and  look  ahead 
to  what  they'd  like  to  leam  next 

How:  time  for  writing  in  learning  logs  is  set 
aside  during  the  last  class  period  of  the  week; 
logs  are  handed  in  to  teachers  who  read 
entries  and  respond  briefly;  multiple  copies 
of  the  log  are  made  and  stapled  together  with 
cover  sheets  to  make  a booklet 

When:  weekly  throughout  the  class  cycle 
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LEARNING  LOG 


Name: Date 

Something  I learned  this  week: 


Something  I eiyoyed  this  week: 


Something  I didn’t  like  this  week: 


What  I want  to  do  next: 
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Math  Activities  Record 

What:  a plan  for  learners'  math  instruction 
and  for  noting  what  got  done,  how  it  went, 
and  any  ideas  for  planning  math  activities 
for  the  next  session 

Why:  to  aid  in  planning  small  group  and 
individual  math  work  and  to  have  a record 
of  learners'  math  work 

How:  can  be  used  by  the  teacher  for  indi- 
vidual learners  or  for  group;  can  also  be 
used  by  learners  to  help  them  keep  track  of 
goals  they  have  set  for  themselves 

When:  before  class  and  at  the  end  of  every 
math  session  during  the  cycle 
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MATH  ACTIVITIES  RECORD 


Name:  Class  Time: 

MATH  ACTIVITIES  PLAN /DATE:  WHAT  HAPPENED: 


Writing  Progress  Checklist* 

What:  form  to  assess  various  aspects  of 
writing  development,  both  in  terms  of 
skills  and  self-concept 

Why:  to  provide  a coherent,  comprehen- 
sive picture  of  a writer' s progress  for  the 
learner  and  for  teachers  to  assess  indi- 
vidual and  group  needs  for  mini-lessons 

How:  can  be  used  by  teacher  after  looking 
at  a writer's  dialogue  journal  and  writing 
folder  and  shared  with  learner  afterwards, 
or  filled  out  with  the  learner 

When:-after  a-few  weeks  of  class  or  when 
the  learner  has  been  writing  for  three  to 
four  weeks;  repeated  and  results  are  com- 
pared at  end  of  cycle 


r 
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* Adapted  and  expanded  from  Sylvia  Greene's  Writing  and  Spelling  Progress  Sheet  from  Basic  Literacy  Kit  (1989) 
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Writing  Progress  Checklist 


Name Date 

Can  Needs  Not  N/A 
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Can 

Do 


B.  Handwriting: 

1.  Prints  upper  case 
lower  case 

2.  Leaves  spaces  between  words 

3.  Optional  Competency:  writes  cursive 
C Spelling: 

1.  Willing  to  use  invented  spelling 


/. 

2 

a 


/ 


* 


2.  Uses  personal  word  dictionary  to  improve 
spelling  and  record  sight  words 

3.  Uses  word  lists/dictionaiy/thesaurus  to 
check  spelling  during  editing  phase 

4.  Writes  letter(s)  that  represent 
consonant  sounds 

& Writes  letters  that  represent  digraphs 
and  blends 

6*  Writes  letter(s)  that  represent  long 
vowel  sounds: 

a.  Uses  a e,  ay,  ai,  to  make  long 
a sound 

b.  Uses  ee,  ea,  to  make  long  e sound 

c.  Uses  i_e,  igh,  y,  to  make  long  i 

d.  Uses  o_e,  oa,  oe,  ow,  to  make 
long  o sound 

e.  Uses  u_e,  ue,  to  make  long  u 

7*  Writes  letters  to  represent  short 
vowel  sounds 


(fid 

b. 

d. 

I 

e 


%.  Applies  the  following  spelling  rules: 

a.  -ft;  -11,  -ss,  -zz 

b.  -k  or  -ck,  -ch  or  tch,  -ge  or  dge 


r. 

8d. 

b. 


Needs 

Work 


Not 

Evident 

sa 


N/A 
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Do 


?.  Uses  "to,  too,  and  two"  correctly 
/a  Uses  "there,  their,'  they’re"  correctly 


9. 

10. 


Needs 

Work 


Not 

Evident 


N/A 


//.  Other  homonyms  used  correctly 
(list  them): 


/#,,  Note  any  spelling  error  patterns 
observed: 


D.  Mechanics: 

1.  Proofreads  first  drafts 

2.  Writes  in  complete  sentences 

3.  Uses  end  punctuation  correctly 

4.  Uses  editing  symbols  to  revise  and 
edit  personal  writing 


//. 


/. 

£ 

J. 

* 


5.  Uses  capital  letters  to: 

a.  Begin  a sentence  Sq. 

b.  Name  people,  places  and  special  things 

c.  Capitalize  the  major  words  in  a title  g 

6.  Uses  commas  to  separate  items  in  a list  ^ 
" around  the  words  people  say 

“ 7 


7.  Uses "_ 
out  loud  " 


8.  Goes  down  to  a new  line  and  indents  when 

beginning  a new  paragraph  2. 

9.  Uses  an  apostrophe  to  show  possession  p 

10.  Uses  an  apostrophe  to  make  a contraction^ 

11.  Uses  verbs  in  the  correct  tense 


ERIC'4<f 


12.  Note  any  other  skills  writer 
has  developed: 
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Writing  Process  Checklist 

What:  form  that  takes  writers  and  teachers 
through  steps  of  writing  process  to  be  used  by 
learners,  teachers,  and  tutors  with  particular 
writing  pieces 

Why:  to  encourage  and  reinforce  indepen- 
dence in  using  the  process  approach  to  writing 

How:  introduced  in  group,  used  individually 
and  stapled  to  drafts  of  piece  being  worked  on ; 
learners  use  independently  or  with  assistance, 
as  needed 

When:  used  with  most  longer  writings  that  go 
through  all  stages  of  writing  process,  espe- 
cially for  new  students,  tutors,  and  earlier 
class  cycles  when  process  writing  is  not 
familiar 


116 


22 


Adventures  in  Assessment,  Volume  2 


Writing  Process  Checklist 

Name: Title: 

* Check  off  each  item  on  the  list  after  you  do  it. 

1.  After  you  write  your  first  draft,  read  it  over  to  yourself. 

2.  Make  any  changes  you  think  are  needed. 

3.  Read  your  piece  to  someone  else  so  they  can  help  you  revise. 

4.  Make  any  changes  you  think  are  needed  to  make  your  piece  clearer. 

5.  Read  the  piece  over  again  yourself,  this  time  to  look  for  things  you  need 
to  edit. 

Is  your  stoiy  in  the  order  that  you  want  it  to  be?  

Do  you  have  complete  sentences?  

Do  you  have  capital  letters  where  you  need  them? 

Do  you  have  the  punctuation  you  need?  (.)  (?)  (!)  

Write  the  words  you  want  to  check  the  spelling  of  here: 


6.  Read  the  piece  to  someone  else  and  use  an  Editing  Checklist. 

7.  You  may  want  to  share  the  piece  with  others.  

8.  Make  a final  copy  
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Writing  Conference  Record 

What:  a form  to  keep  track  of  what 
happens  with  an  individual  learner  during 
informal  writing  conferences 

Why:  to  maintain  a cumulative,  working 
record  of  skills  worked  on  by  each  learner 

How:  teacher  writes  brief  notes  after 
writing  conference  or  other  significant 
interaction  with  learner  during  writing 
time;  form  is  stapled  to  learners'  writing 
folders 

When:  each  time  teacher  or  tutor  works 
with  learner  individually  on  particular 
writing  skills;  cumulative  records  become 
part  of  learners'  assessment  portfolios 
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WRITING  CONFERENCE  RECORD 


Writer’s  Name: 

Title:  Skills  Worked  On:  Date: 
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Spelling  Self-Test 

What:  a form  to  help  learners  focus  on  a 
reasonable  number  of  words  to  study  and 
test  themselves  on,  if  desired 

Why:  to  meet  learners'  need  to  focus  on 
spelling  improvement  by  providing  a for- 
mat to  encourage  learners  to  select  words 
to  leam  to  spell  from  their  own  writing 

How:  learners  use  forms  independently, 
sometimes  in  conjunction  with  group  les- 
sons on  ways  to  study  spelling;  learners 
may  have  another  learner  "test"  them  by 
dictating  their  words  to  them;  correct 
spellings  are  copied  on  left  hand  list,  then 
folded  over  for  self-test  on  the  right  side; 
learner  checks  own  work  by  by  unfolding 
the  page 

When:  depends  on  learners'  needs;  usu- 
ally, lists  of  seven  or  fewer  words  are 
studied  for  a week;  form  used  as  needed 
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SPELLING  SELF-TEST 


Name: Date: 


WORDS  FROM  PERSONAL  DICTIONARY/  SELF-TEST: 

WRITING: 

1. 1. 

2.  2. 

3.  3. 

4.  4. 

5.  5. 

6.  6. 

7.  7. 
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Dialogue  Journals 

What:  simple  yet  powerful  tools  for  developing  learners'  confidence 
and  literacy  skills;  agreat  way  for  teachers  and  learners  to  get  to  know 
each  other.  Dialogue  journals  are,  as  their  name  implies,  a conver- 
sation on  paper  between  two  people. 

Why:  to  provide  an  opportunity  for  reading  and  writing  for  commu- 
nication; to  act  as  a "warm-up"  to  writing  in  the  sense  that  no 
judgements  or  corrections  are  made.  Despite  the  lack  of  editing, 
journal  entries  over  time  almost  always  show  progressive  develop- 
ment toward  more  conventional  spelling,  punctuation,  and  grammar. 
This  is  probably  due  to  many  factors,  one  of  which  may  be  the 
teachers 's  modelling  of  conventional  spelling,  etc.  Dialoguejoumals 
are  also  places  where  issues  and  problems  that  are  bothering  learners 
can  be  safely  aired  and  not  lost  in  the  shuffle. 

How:  journals  are  made  of  lined  paper  stapled  together  with  colored 
paper  covers.  Each  teacher  writes  to  a group  of  learners,  ending  the 
entry  with  a question  for  the  learner  to  respond  to  in  writing . Journals 
are  left  in  students’  mailboxes.  Learners  read  their  journals  on  their 
own  if  they  can,  get  help  with  the  reading  if  needed,  and  respond  in 
writing  with  teacher’ s help  if  needed.  Learners  are  asked  to  end  their 
entries  with  a question  for  the  teacher  and  put  the  j oumals  in  teachers ' 
mailboxes. 

When:  can  be  incorporated  into  any  program  at  any  time.  We  use  the 
first  20  minutes  or  so  of  each  class  period  for  dialogue  journals  since 
there  are  often  latecomers  and  journal  writing  is  an  individual 
activity.  It  also  works  as  a good  "warm  up"  activity  for  further 
reading  and  writing.  Teachers  respond  in  the  journals  before  the  next 
class;  this  sometimes  means  taking  the  journals  home.  We  rotate  the 
groups  of  learners  which  teachers  write  to  every  eight  weeks  or  so. 
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Sample  of  a dialogue 
journal. 

This  first  entry  is  from  the 
first  day  of  class. 
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This  entry  is  from  a 
class  about  six  and 
a half  months  later. 
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Janet  Kelly  is  an  ABE  literacy  teacher  and  project 
director  at  Read/Write/Now  in  Springfield. 


Note 

All  of  the  materials  included  in  this  article,  as  well  as 
other  curricuulum  materials,  can-also  be  found  in~ 
the  ABE  Language  Arts  Curriculum  Kit  sponsored 
by  SABES.  The  0-4  Kit  is  available  through  the 
SABES  Regional  Support  Centers  at  Holyoke 
Community  College  (Holyoke),  Quinsigamond 
Community  College  (Worcester),  Bristol  Com- 
munity College  (Fall  River),  Northern  Essex  Com- 
munity College  (Lawrence),  and  the  Adult  Lit- 
eracy Resource  Institute  (Boston). 
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Self-Assessment: 
Doing  and  Reflecting 


Paul  Trunnel 


| his  article  focuses  on  self-assessment  and 
| the  activation  of  the  learner.  A major 
| assumption  behind  self-assessment  is  that 
learners  are  capable  of  evaluating  themselves,  that 
they  have  explicit  goals,  and  that  a process  of  self- 
evaluation  will  make  them  more  motivated  and 
aware  of  their  own  learning  process.  This  process 
can  make  learning  more  meaningful  and  encourage 
learners  to  become  active  in  shaping  their  own 
learning  environment. 

The  Context 

The  Harborside  Community  Center,  in  East 
Boston,  provides  free  ABE  and  ESL  classes  to  an 
increasingly  diverse  population,  including  Latinos, 
Cambodians,  Vietnamese,  Jamaicans,  Cape 
Verdeans,  and  native  bom  North  Americans  (many 
of  Italian  descent). 

The  ABE  2 class  that  I teach  has  about  sixteen 
students  and  serves  as  the  pre-GED  and  pre-EDP 
course.  About  half  the  students  are  native  English 
speakers.  The  class  meets  three  times  a week  to 
study  reading,  writing,  and  math.  The  diversity  of 
skill  levels  in  the  class  makes  it  a kind  of  one-room 
schoolhouse:  a few  students  have  reading  difficul- 
ties; many  of  the  students  have  language  issues 
specific  to  learning  English  as  a second  language; 
a few  students  will  soon  be  taking  their  GED’s  or 
enrolling  in  the  EDP  class.  Students  stay  in  my  class 
anywhere  from  three  to  18  months. 


Much  of  the  assessment  that  takes  place  at 
Harborside  falls  into  the  “alternative”  category. 
At  the  school's  Open  House,  prospective  ABE 
students  place  themselves  in  classes.  Learning 
questionnaires,  a self-evaluatory  math  sheet,  self- 
selected  reading  and  writing  activities,  and  discus- 
sions with  teaching  staff  help  learners  through  this 
process,  (appendix  2)  In  the  ABE  2 classroom  we 
use  goals  surveys  and  activities,  learning  behavior 
questionnaires,  learning  contracts,  and  regular  con- 
ferences to  assess  needs  and  determine  progress. 

All  of  the  different  assessment  tools  that  we  use 
have  developed  through  a process  of  trial  and  error. 
My  goal  has  always  been  to  fit  the  activity  to  the 
learning  needs  and  styles  of  the  students,  and  also 
to  my  own  teaching  style.  I think  this  process  of 
experimentation  is  vital.  The  manner  in  which  we 
assess  ourselves  and  the  questions  which  we  ask 
strongly  influence  what  we  study  and  how  we  study. 
The  activities  that  I discuss  in  this  article— the  skills 
evaluation  sheet  and  the  learning  log-are  two  of  the 
first  alternative  assessment  activities  that  I tried  in 
my  class.  Their  development  exemplifies  this 
process  of  experimentation. 

Developing  the  Skills  Evaluation  Sheet 

The  skills  evaluation  activity  evolved  while  I 
was  looking  for  an  ongoing  assessment  tool  I could 
use  with  the  whole  class.  At  the  time,  I didn’t  have 
anything  other  than  some  goals-oriented  activities 
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FIGURE  1 

which  seemed  too  individualized  to  do  as  a class. 
Using  the  suggestions  of  Nunan  (1988)  and  Auerbach 
(1990)  as  my  guides,  I sketched  out  several  differ- 
ent sheets,  with  successively  fewer  and  fewer 
instructions  because  my  class  was  impatient  with 
long-winded  explanations. 

The  final  form  was  an  almost  blank  piece  of 
paper  with  just  enough  information,  I hoped,  to 
point  an  uncertain  learner  in  the  right  direction.  The 
inspiration  had  come  from  the  forms  and  ideas  of 
Nunan  and  Auerbach,  but  the  final  version  was 
tailor-made  for  my  class,  (figure  1,  appendix  3 ) 

My  goal  was  to  keep  the  form  and  the  activity 
from  being  “top-heavy.  ” This  seems  to  be  the  case 
with  many  of  the  forms  and  activities  that  I have 
read  in  alternative  assessment  literature.  This  is  not 
meant  to  be  a mark  against  these  forms;  I simply 
mean  that  learner  centered  education  needs  to  be 
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student  and  context  specific.  For  me  this  means  that 
much  of  the  material  on  the  forms  that  have  inspired 
me  seems  irrelevant  to  my  ABE  2 class  and  its 
. members. 

Using  the  Skills  Evaluation  Sheet 

When  I first  handed  out  the  skills  evaluation 
sheet,  I explained  what  the  columns  were  for.  The 
skills  column,  I said,  is  for  making  a list  of  all  the 
different  skills  and  activities  we  have  done  since  the 
last  time  we  evaluated.  These  may  include  frac- 
tions, subtraction,  learning  how  to  use  commas, 
writing  essays  on  learning,  a trip  to  the  library,  etc. 
Much  of  what  is  included  in  this  column  may  be 
specialized  because  ofthe  individual  learning  projects 
of  many  students.  Then,  as  a class,  we  made  a list 
on  the  board,  covering  as  specifically  as  possible 
(e.g.,  “fraction  reducing”  or  “fraction  subtrac- 
tion” instead  of  just  “fractions”)  all  the  skills  and 
activities  we  had  done. 

The  second  evaluation  column,  I explained,  is 
to  put  what  students  think  about  the  different 
activities  and  skills  we  have  covered.  I usually  need 
to  explain  what  I mean  by  evaluation.  For  starters, 
we  talk  about  different  systems  or  words  we  can  use 
to  evaluate.  For  example,  a 1 - 5 best  to  worst 
listing;  or,  I learned  it/I  need  more;  or  more 
specialized  responses,  such  as  “I  liked  it  but  I don’t 
remember  it  too  much.”  I leave  this  up  to  the 
student,  rather  than  providing  the  system  or  the 
words,  because  I am  interested  in  individualized 
answers.  Also,  each  student  has  the  opportunity  to 
choose  a different  system,  which  empowers  them 
and  gives  them  the  opportunity  to  explore  evalua- 
tion methods  that  best  suit  them. 

Then,  we  proceed  to  the  back  page.  I ask  the 
learners,  now  that  they’ve  listed  and  commented  on 
all  of  these  items,  to  identify  a few  that  they  feel 
they  have  learned  and/or  they  still  want  to  study. 
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Also,  I tell  them  to  include  other  subjects  they  might 
want  to  touch  on  in  the  future. 

Throughout  the  activity  of  filling  out  the  sheets, 
we  discuss,  as  a group,  the  different  skills,  activi- 
ties, and  systems  that  we  come  up  with.  This 
ussually  leads  us  to  conclude  with  a good,  affirming 
sense  of  what  we’ve  accomplished,  as  well  as  a less 
muddled  vision  of  what  we  are  going  to  continue 
studying  and  where  we  will  go  next. 

Some  students,  particularly  those  who  advocate 
more  traditional  methods  of  assessment  and  learn- 
ing, have  write  very  little;  but,  the  class  discussion 
seems  to  involve  them  effectively.  A few  students 
surprised  me  in  their  comments  about  areas  they 
hoped  to  review— areas  I thought  they  had  a firm 
grasp  on.  Overall  it  seemed  that  the  form  accom- 
plished the  goal  of  focusing  the  class  for  the 
remainder  of  the  cycle. 

After  the  exercise,  I asked  students  what  they 
thought  of  it.  In  general,  they  liked  it.  One  student 
suggested  we  do  it  every  month;  other  students 
agreed.  I suggested  that  we  do  it  once  a week,  but 
the  students  thought  that  would  be  too  often. 
Another  student,  said,  “You  already  know  what  I 
know.  What’s  the  use?”  I explained  that  I didn’t 
know  all  he  knew;  and  particularly,  I had  no  idea 
about  how  he  thought  he  was  doing.  I told  him  that 
was  important  to  me. 

All  in  all,  the  sheet  seems  effective  in  assessing 
group  progress  and  encouraging  students  to  reflect 


. . .Ac  thmf  develop,  journal*  provide  con  erf  avidanc*  of  *tud*nt*‘ 
prograss.  Tsacbars  can  evaluate  thea  in  bams  of  criteria  libs 
range  of  topics /con tent,  elaboration  of  ideas  (including  use  of 
details,  axaag>lea,  depth  of  analysis,  emotional  force,  etc.), 
length  of  entries,  grammatical  development  (specific  forms  libs 
tense  markers,  fragments,  etc.)  and  coherence  as  well  as  students' 
own  perspective  on  their  learning.  Students  can  use  them  for  self- 
evaluation  by  reading  and  responding  to  the  finished  products, 
noting  changes  and  areas  needing  work.  (Auerbach) 


FIGURE  2 


on  their  individual  progress.  It  allows  me  to  see 
what  was  effective,  popular,  or  not  and  works  very 
well  in  telling  us  where  we  need  to  go  next. 

This  device  is  intensive  and  comprehensive 
enough  that  we  will  use  it  maybe  once  every  four  to 
six  weeks,  as  a good  grounding  activity  for  the 
class.  In  the  future,  I might  have  learners  use  their 
progress  portfolios  as  a means  to  inspire  the  list, 
instead  of  prompting.  This  way,  as  well,  a more 
personalized  list  may  develop. 

Developing  the  Learning  Log 

While  the  skills  evaluation  sheet  was  a helpful 
tool  for  seeing  progress  over  a month  or  six  weeks, 
I still  wanted  an  activity  for  more  frequent  self- 
assessment.  I adapted  an  idea  for  a weekly  learning 
log  from  the  weekly  progress  sheets  suggested  by 
Nunan  and  Auerbach  and  the  weekly  reviews 
suggested  by  Lawson,  (figure  2 and  3).  A learning 
log  is  a good  example  of  immediate  self-assess- 
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ment.  Students  are  asked,  at  certain  intervals—daily 
or  weekly~to  reflect  and  record  in  writing  what 
they  have  learned.  Learning  logs  can  serve  many 
purposes. 

As  dialogue  journals:  Students  who  are  less 
comfortable  with  writing  are  given  a purpose  and 
encouragement  to  put  their  thoughts  into  words  in 
a less  threatening  manner.  The  writing  can  be 
treated  as  a dialogue,  meaning  that  what  students 
write  down  will  be  responded  to. 

As  classroom  ritual:  Once  every  week  or  two 
weeks,  or  after  particular  events,  students  can  count 
on  this  method  as  a means  of  evaluating  their 
experience.  By  making  it  a regular  event,  it  be- 
comes endowed  with  greater  authority,  and  in  this 
instance,  that  authority  is  bestowed  on  learners’ 
reflections.  In  this  way,  the  learning  logs  become  a 
respected  means  of  evaluation,  in  place  of  an  exam 
or  paper. 

As  a review:  Students  may  write  about  specific 
ideas  or  terms  they  have  learned.  Again  this  may 
substitute  for  a quiz,  or  be  used  as  a study  notebook 
for  one.  Often  adult  students  are  not  in  the  habit  of 
keeping  notebooks  or  organizing  their  learning 
materials.  Journals  can  help  sort  things  out.  Also, 
at  least  in  my  class,  the  learning  we  do  is  often 
extemporaneous:  a student  might  ask,  “What  is 
area?”  and  a twenty  minute  lesson  in  area  will 
follow.  Learning  logs  can  be  the  place  to  record 
some  of  those  concepts. 

As  a question-and-answer  period:  Students 
may  choose  to  ask  their  teacher  questions  in  the  log 
about  a subject,  or  about  the  class  in  general,  that 
they  might  not  have  time  to  ask  in  class . The  reverse 
is  true:  I can  ask  the  students  questions  about 
whether  or  not  they  understand  something.  I may 
not  have  had  time  during  the  lesson  to  find  out 
where  everyone  was;  or  I may  just  prefer  a more 
private  means  of  communication. 


Again,  I changed  the  format  to  fit  my  teaching 
style  and  the  needs  of  the  class.  The  ESL-oriented 
questions,  for  example,  “what  new  words  have  I 
learned  this  week?”  seemed  inappropriate  for  the 
level  of  my  class.  In  fact,  I chose  to  eliminate  all  the 
questions:  I still  wanted  to  explore  the  possibilities 
of  a loosely-directed  task,  rather  than  a more 
structured  task  like  a questionnaire.  It  was  my  hope 
that,  by  simply  giving  them  a book  and  asking  them 
to  write  about  their  learning,  their  entries  would  be 
more  self-exploratory  and  therefore  more  valuable 
to  the  students  and  to  me. 

Using  the  Learning  Log 

I began  using  learning  logs  at  the  end  of  math 
class.  I asked  students  to  record  in  a little  blue  exam 
book  what  work  they  had  done  and  if  they  had  any 
questions.  The  purpose  was  two-fold:  first,!  wanted 
a specific  record  of  the  work  they  had  been  doing; 
second,  I wanted  to  start  using  this  as  a learning  log/ 
dialogue  journal.  The  entries  varied  in  interest  and 
information  provided: 

“Number  Power  2,  pg.  34” 

“Today  I worked  on  my  multiplying  and 
carrying.” 

“I  done  QK~  some  of-them.  ” 

“I  worked  on  fractions  page  84  (book) 
Building  Basic  Skills  in  Math.  I am  doing 
alright  But  I need  some  help  to  get  going. 

“I  started  today,  doing  integers.  I under- 
stand it  pretty  much.  While  I was  Brushing 
up  on  them,  I had  some  trouble  in  doing  the 
fractions.  I would  like  to  start  learning  the 
fractions  next  week.  I will  need  some  great 
deal  of  help  on  them.  Thanks.” 

Only  the  last  two  learners  seem  to  be  fully 
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engaged  in  the  idea  of  the  learning  log.  However, 
there  is  value  in  even  the  abbreviated  responses. 
Obviously,  for  some  students,  the  process  of  writ- 
ing in  the  log  was  language  practice.  For  others,  it 
at  least  met  the  first  goal  of  the  exercise:  creating  a 
record  of  the  work  they  had  done,  (figure  4) 

The  second  time  I passed  the  little  blue  books 
out,  I asked  everyone  to  take  a moment  and  reflect 
on  the  learning  they  had  done  in  the  previous  week. 
The  entries,  I said,  could  be  anywhere  from  a 
sentence  to  whatever  length  they  felt  necessary.  We 
took  a few  minutes  to  discuss  the  purpose  of  the 
logs.  In  the  discussion,  learners  brought  up  issues 
they  had  with  this  approach  to  assessment:  “What 
do  you  want  us  to  write?  Why?  We  haven’t  learned 
anything.  This  is  a waste  of  my  time.”  I tried  to 
explain  that  this  was  a way  of  recording  progress, 
and  an  opportunity  to  start  a dialogue  on  learning. 
Yes,  it  is  true  that  over  the  last  week  we  have  not 
done  much  in  the  way  of  skills  exercises.  But,  what 
about  the  trip  to  the  library?  What  about  the  class 


discussion  we  had  about  AIDS? 

The  journals  were  filled  out  and  turned  in 
mostly  with  short,  unreflective  entries  such  as: 

“We  learned  about  the  Constitution  about 
Civil  Rights.” 

But,  another  student  had  this  to  say: 

“As  you  can  tell,  it’s  a bit  confusing  with 
integers.  But  I do  understand  the  basics,  as 
long  as  it  relates  to  money.  Anyhow,  during 
the  past  week  I have  been  learning  on  my 
own,  on  fractions.  I think  so  far  I’m  doing 
good  considering  I had  no  clue  what  was 
going  on  with  fractions.  The  workbook  you 
have  let  me  borrow  (Number  Power  2)  is 
quite  helpful  and  easy  to  understand  step  by 
step.  If  I need  your  help  I’ll  let  you  know.  ” 

Later,  I had  a conversation  with  the  student  who 
wrote  only  one  sentence.  We  talked  about  how 
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limited  class  hours  were  and  how  there  was  only  so 
much— skills-related  or  otherwise— that  we  could 
cover.  She  felt  the  class  discussions  we  had  had 
about  the  constitution,  about  drugs,  and  about 
AIDS  were  not  learning.  I took  this  as  a cue  that  I 
should  connect  our  discussions  more  firmly  with  the 
skills  she  wants  and  needs  to  achieve  her  GED.  But 
she  also  acknowledged  her  impatience  as  a source 
of  frustration  and  told  me  that  she  took  an  hour  or 
two  at  home  every  night  to  do  work  in  her  work- 
book. In  the  following  class,  she  made  a suggestion 
pertaining  to  math  class  that  was  helpful  and 
thoughtful,  and  which  we  put  into  immediate  use. 
Finally,  a few  days  later,  when  she  was  having 
difficulty  with  her  math,  she  said,  “Give  me  my 
learning  log,  I’m  having  a hard  time.  ” So,  even  her 
frustration— and  my  frustration  with  her  frustra- 
tion-had a silver  lining. 

In  the  future,  I hope  to  make  using  learning  logs 
a regular  “ritual”  in  the  classroom  as  Lawson 
(1988)  and  Auerbach  (1990)  have  recommended. 
At  the  end  of  every  week,  or  even  every  day,  we 
could  take  five  or  ten  minutes  to  reflect  on  the 
learning  we  have  done  over  the  week.  At  the  end  of 
the  week  we  might  take  a few  minutes  to  plan  out 
some  goals  for  the  week  to  come.  Also,  since  some 
students  had  difficulty  with  the  lack  of  structure  of 
the  assignment,  I might  re-introduce  some  of  the 
questions  provided  by  Auerbach,  Lawson,  and 
Nunan. 

Paul  Trunnel  teaches  at  the  Harborside  Community 
School  in  East  Boston. 
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When  Asking  Isn't  Enough 


Kathy  Brucker 


|S|||||  his  year  I tested  a variety  of  evaluation 
f|Bf  tools  with  my  level  one  Spanish-speaking 
. Pv  students  at  El  Centro  del  Cardenal  in 
Boston.  These  evaluations  were  based  on  our  class 
lessons,  but  they  could  be  adjusted  for  any  class  and 
used  at  any  time. 


First  Evaluation 

The  first  evaluation  was  completed  by  students 
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at  the  end  of  the  first  week  of  classes.  I gave  the 
students  20  minutes  and  provided  activities  for 
those  who  finished  early,  (figure  1) 

I encouraged  the  students  to  answer  this  evalu- 
ation in  Spanish.  Although  I don’t  read  Spanish 
well,  I can  always  find  a staff  person  or  another 
teacher  to  translate  for  me.  This  extra  step  is  worth 
my  time  because  students  can  give  me  “true,”  that 
is  to  say,  more  accurate  and  more  complete  an- 
swers, using  their  first  language.  They  feel  freer  to 
express  themselves  and  let  me  know  what  they 
want.  It  also  gives  me  a sense  of  the  students’  native 
language  competency.  This  is  important  in  order  for 
me  to  understand  where  students  are  starting.  I ask 
students  to  identify  themselves  on  the  assessment  so 
that  I can  find  out  if  someone  is  uncomfortable  with 
certain  activities,  such  as  going  to  the  board, 
reading  out  loud,  being  on  videotape,  or  working 
with  partners. 

Second  Evaluation 

The  next  type  of  evaluation  I used  helped 
students  identify  which  activities  they  had  enjoyed 
and  found  useful  and  which  they  had  not.  This  was 
done  a week  later  (two  weeks  into  the  cycle)  and 
took  about  one  half  hour,  (figure  2) 

I used  the  headings  “a  little,”  “ok,”  and 
“much  (a  lot),”  because  this  is  language  the 
students  know  and  can  read.  I also  displayed 
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examples  of  each  activity  mentioned  in  case  stu- 
dents had  trouble  connecting  the  written  description 
with  the  activity. 

The  problem  with  this  evaluation  was  that  it 
didn’t  give  me  enough  direction.  Because  everyone 
had  different  opinions,  it  was  hard  to  integrate  all 
individual  tastes.  One  way  I would  amend  the 
second  evaluation  is  by  adding  the  following  ques- 
tions: 

• Which  did  you  enjoy?  Why? 

• Which  part  helped  you  remember  English? 

• Which  didn’t  you  enjoy?  Why? 

I think  these  questions  would  provide  better 
feedback  because  they  ask  students  to  reflect  on  why 
they  enjoy  certain  activities  and  why  they  don’t 
enjoy  others.  This  would  give  me  a better  sense  of 


what  activities  are  useful  and,  at  the  same  time,  help 
students  reflect  on  their  learning  styles. 

Third  Evaluation 

The  next  evaluations  incorporated  some  learn- 
ing strategies  to  encourage  critical  thinking  and 
students  taking  control  of  their  own  learning.  This 
evaluation  took  students  up  to  one  hour  to  under- 
stand and  complete,  (figure  3) 

Most  of  the  answers  were  predictable:  they 
liked  learning  with  pictures  and  having  conversa- 
tions in  English.  But,  there  were  surprises,  like  the 
student  who  said  he  only  wanted  to  speak  and  hear 
English,  yet  never  spoke  English  in  the  classroom 
and  often  asked  for  translations. 

Part  of  the  third  evaluation  was  taken  from  El 
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Centro’s  teacher  evaluation  that  all  students  com- 
plete at  the  end  of  each  cycle.  I like  to  include  this 
section  to  see  how  students  rate  themselves  on  the 
same  type  of  scale  as  they  rate  the  teacher.  Because 
there  is  a point  in  each  cycle  when  students  question 
whether  they  are  making  progress,  I also  like  to 
include  statements  that  help  them  consider  how 
much  they’ve  learned.  In  addition  to  helping  stu- 
dents see  what  we  have  covered,  this  evaluation 
gives  them  the  opportunity  to  request  individual  or 
class  reviews  of  particular  activities. 

Subsequent  biweekly  evaluations  were  format- 
ted with  the  addition  of  new  activities.  I found  this 
format  the  most  accessible  to  the  students  and  the 
one  we  could  draw  useful  conclusions  from  the 
most. 

Final  Evaluation 

The  final  evaluation  was  done  in  English.  I had 
assumed  the  students  would  use  Spanish,  but  when 
I gave  them  the  evaluation,  they  started  writing  in 
English;  of  course,  I didn't  stop  them!  I instructed 
them  not  to  sign  the  evaluation  if  they  wished  not  to 
be  identified.  Students  spent  from  one  to  one  and  a 
half  hours  on  this  evaluation,  (figure  4) 

I included  a question  to  help  the  students  begin 
to  think  about  their  learning  strategies.  I hoped  that 
this  evaluation  would  enable  the  students  to  tell  their 
next  teachers  how  they  best  learn  and  in  what  ways 
the  teacher  could  help  them.  Next  time,  I plan  to 
introduce  the  idea  of  learning  strategies  early  in  the 
cycle  so  that  students  can  experiment  with  learning 
styles  throughout  our  time  together. 

Analyzing  Our  Learning 

I find  evaluations  very  helpful.  In  addition  to  the 
roles  they  can  play  as  described  above,  I now  also 
use  them  to  promote  discussions  about  tensions 
people  are  feeling  in  the  classroom.  For  example, 
working  with  a homogeneous  language  group  I find 


people  disagreeing  on  how  much  native  language  or 
English  should  be  used.  I read  in  evaluations  the 
frustrations  students  are  feeling  about  anything 
from  having  enough  English  conversation  in  class 
to  personal  problems  with  neighbors.  Using  the 
evaluation  as  a code  to  promote  discussion  enables 
me  not  only  to  introduce  new  phrases  and  vocabu- 
lary to  students  but  also  provides  an  opportunity  to 
have  them  role  play  situations  they  want  to  discuss. 

I like  to  know  which  lessons  are  successful  and 
which  are  not,  and  what  studetns  consider  improtant 
or  enjoyable.  Most  of  the  time,  asking  isn't  enough. 
I find  that  writing  about  our  learning  helps  us 
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analyze  it.  Evaluations  give  the  students  input  and 
control  over  their  classroom.  They  help  people 
structure  their  thoughts  and  discover  if  they're 
learning,  what  they're  learning,  how  they're  learn- 
ing, and  finally,  why  they're  learning. 

Kathy  Brucker  is  an  ESL  teacher  at  El  Centro  del 
Cardenal  in  Boston  and  Concilio  Hispano  de  Cam- 
bridge. 
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What  You  See:  Ongoing  Assessment 
in  the  ESL/Literacy  Classroom 


Janet  Isserlis 


his  essay  describes  ongoing  action  research1 
ion  how  adult  ESL/  literacy  learners  progress 
/vv^^lin  and  out  of  the  classroom.  Its  purpose  is 
to  describe  ways  in  which  facilitators  understand 
and  describe  learning  and  work  with  adults  to 
increase  their  awareness  of  how  they  learn.  It  is  my 
hope  that  sharing  these  thoughts  with  other  literacy 
practitioners  may  encourage  them  to  adapt,  expand , 
and  further  develop  these  ideas  in  ways  that  make 
sense  within  their  own  contexts. 

Background 

With  the  current  widespread  focus  on  assess- 
ment-as  with  every  educational  trend— the  poten- 
tial for  growth  as  well  as  abuse  exists.  (Think  about 
empowering  learners  through  a basal  whole  lan- 
guage series  and  understand  how  contradictions 
develop  in  the  name  of  commerce.)  Publishers  have 
begun  to  jump  on  the  progressive  assessment  band- 
wagon and  have  commodified  “new”  assessment 
practices.  Nonetheless,  one  very  positive  outcome 
in  this  recent  resurgence  of  attention  to  assessment 
is  its  bringing  to  awareness  real  links  between 
assessment  and  learning. 

Assessment  is  observation  and  asking  ques- 
tions. The  reports  we  send  to  funders  are  related  to 
the  outcomes  of  some  of  those  questions  but  are 
removed  to  some  extent  from  the  everyday  business 
of  working  in  a classroom.  Assessment  is  interwo- 
ven in  this  daily  work  in  obvious  as  well  as  subtle 


ways.  When  a learner  says,  “I  don’t  understand,” 
we  know  that  she  needs  more  information  or  needs 
to  have  something  rephrased  or  re-explained.  This 
kind  of  spontaneous  evaluation  of  a ieamers  needs 
is  the  most  obvious  form  of  ongoing  assessment. 
When  another  learner  consistently  asks  questions 
about  grammar  over  a period  of  time,  we  also  learn 
something  about  that  learner’s  thinking  and  learn- 
ing processes  and  possibly  about  that  learner’s 
previous  experience  with  formal  education.  When 
a learner  always  copies  everything  from  the  black- 
board, breaks  lines  in  her  notebook  in  exactly  the 
same  way  as  they’re  broken  on  the  board  (even  if 
she  has  enough  space  in  her  notebook  not  to  break 
the  lines),  we  learn  something  about  how  she 
understands  print. 

The  assessment  questions  we  ask  range  from  the 
very  broad  to  the  very  specific.  Broader  questions 
frame  our  thinking  about  how  we  know  what  we 
know  about  learning,  but  they  are  not  particularly 
helpful  without  specific,  local  questions  about 
particular  learners  in  concrete  contexts.  We  might 
begin  by  asking  what  we  mean  by  assessment. 
Assessment  of  what?  of  whom?  When  does  this 
assessment  occur  and  why?  If  we  decide  that  we 
assess  learners  to  satisfy  funders,  then  we  remove 
learning  from  the  equation  to  some  degree.  We 
want  to  be  able  to  show  the  people  who  pay  us  that 
their  money  is  being  well  spent.  So,  at  worst,  we 
may  teach  to  a standardized  test,  showing  low  pre- 
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and  high  post-  test  scores.  We  may  tell  learners  that 
this  is  just  a test,  and  work  towards  it  “on  the  side” 
but  use  the  rest  of  our  valuable  classroom  time  on 
“real”  learning  and  teaching.  In  other  instances 
we  may  try  to  educate  funders  and  tell  them  that  we 
use  portfolio  assessment— that  we  look  at  learners’ 
progress  over  time  in  various  areas— through  dia- 
logue journals,  writing  samples,  and  taped  readings 
done  at  various  intervals  during  the  course  of  a 
program.  We  may  ask  learners  to  report  on  their 
own  progress  and  document  their  responses  to 
surveys  we  conduct  throughout  the  year.  Or  we 
may  use  any  combination  of  the  above. 

These  are  product-oriented  views  of  assess- 
ment, and  the  concerns  of  funders  are  very  real 
problems.  Nonetheless,  for  this  discussion,  I would 
like  to  set  aside  these  funders  and  their  needs  and 
return  to  learners  and  theirs. 

Assessment  Every  Day 

Ongoing  assessment  has  to  do  with  classroom 
activity  that  occurs  continuously.  Such  ongoing 
assessment  has  less  to  do  with  written  reports  and 
far  more  to  do  with  the  interactive,  dynamic, 
dialogic  roles  of  both  teachers  and  learners.  It  has 
to  do  with  responding  to  learners’  questions  every 
day  and  with  actively  noting  the  kinds  of  questions 
learners  ask,  the  ways  in  which  learners  respond  to 
print  and  oral  communications,  the  kinds  of  mis- 
takes they  make,  the  ways  in  which  they  go  about 
correcting  their  own  mistakes,  and  the  ways  in 
which  their  classmates  might  correct  them.  This 
kind  of  ongoing  observation  and  assessment  is 
inseparable  from  good  teaching  practice. 

In  1988,  literacy  practitioners  at  the  Interna- 
tional Institute  of  Rhode  Island  began  a process  of 
documenting  classroom  activity  in  order  to  develop 
a vehicle  for  recording  ongoing  assessment.2  Over 
time,  we  came  to  notice  not  only  what  was  happen- 
ing with  particular  learners  but  also  ways  in  which 


to  examine,  frame,  and  question  ongoing  events  in 
general.  This  process  is  described  at  length  else- 
where3; what  emerged  from  the  process  has  been  a 
heightened  awareness  on  the  part  of  teaching  staff 
of  the  importance  of  ongoing  critical  reflection  and 
interaction  with  learners.  Active  observation  and 
attentive  listening  enable  practitioners  to  go  beyond 
the  more  evident  aspects  of  learners’  progress— 
e.g.,  increases  in  length  of  journal  entries,  im- 
provements in  spelling— and  become  more  closely 
aware  of  the  processes  through  which  learners 
acquire  language  and  literacy. 

For  Example 

To  illustrate  the  kinds  of  daily  interactions  and 
specific  instances  which  teach  us  about  how  learn- 
ers learn,  the  following  anecdotal  accounts  describe 
adult  ESL/literacy  learners  and  ways  in  which  they 
help  themselves  and  help  me  to  learn  about  their 
learning  processes. 

Rosalie  has  been  in  America  for  about  18  years. 
She  was  bom  in  Haiti  and  raised  in  the  Bahamas.  At 
present  she  works  as  a dishwasher.  She  came  to 
class  with  the  ability  to  speak  and  understand 
English  and  to  write  her  name.  She  knew  the  names 
of  the  letters  of  the  alphabet  but  had  no  strong  sense 
of  sound/symbol  correspondence.  Even  five  to 
seven  line  language  experience  activities  (LEA) 
seemed  to  overwhelm  her.  She  could  “read”  the 
stories  at  the  blackboard  after  several  other  learners 
had  read  them— she  was  able  to  memorize  much  of 
the  passage  and  was  also  very  willing  to  ask  for 
assistance  when  she  couldn’t  decode  a word.  When 
learners  were  reading  the  typed  versions  of  the 
previous  day’s  LEA,  I gave  Rosalie  different  col- 
ored highlighter  pens,  asking  her  to  highlight  all  the 
“b”  words  in  one  color,  all  the  same  words  in 
another  (“and”,  “name,”  etc.).  Her  ability  to 
memorize  was  evident,  for  example,  when  she 
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Monday,  November  4 

16  students/R's  daughter 

R came  back;  she  still  sounds 
congested  and  says  she's  tired. 
She  didn ' t bring  the  kids . Re- 
viewing the  homework  took  the 
entire  hour.  Everyone  read  the 
story  aloud — this  took  us  through 
the  first  hour.  After  the  break, 
I asked  Yvette  to  read,  which  she 
did,  sort  of,  with  everyone  of  us 
also  reading  along.  I asked  if 
people  thought  it  was  good  or  not 
to  have  each  person  read  a story 
— because  people  do  fall  into 
chatting  to  each  other  when  oth- 
ers are  reading.  Margaret  didn't 
even  wait  for  me  to  finish  asking 
the  question;  she  was  adamant 
( . . . Jthat  hearing  everyone  read 
helps  her — particularly,  I think, 
with  pronunciation.  She  said 
that  was  only  her  belief;  no  one 
seemed  to  express  an  opposing 
opinion,  but  I did  suggest  that 
if  people  didn't  want  to  listen 
to  everyone  read  that  they  could 
do  things  on  their  own. 

It  seems  that  when  we  have  a 
particularly  guided  structure 
that  the  energy  is  up  and  people 
are  beginning  to  interact  a bit 
more  widely,  beyond  their  immedi- 
ate tables /spheres  of  friends. 

I asked  L to  lay  off  P a 
little  bit;  I wasn't  particularly 
heavy  handed  about  it  with  him 
but  he's  teasing  her  in  a way 
that  I think  will  make  her  uncom- 


fortable (if  it  isn ' t already ), 
calling  on  her  to  read,  pointing  at 
her,  etc.  He's  being  a little 
intense  with  her  — behaving  dif- 
ferently from  the  kind  of  teasing/ 
sparring  he  engages  in  with  Rany. 

I hope  this  will  go  away. 

More  volunteers  to  write;  Yvette 
even  went  to  copy  "yes'  from  her 
paper.  She  did  her  homework  and 
said  she'd  done  it  alone.  Most  of 
the  handwriting  was  hers,  but  some 
questions  had  clearly  been  an- 
swered/written by  someone  else. 
Marco  was  trying  to  tell  me  that 
[Yvette]  had  confused  "don't  know" 
with  "no,"  I think,  but  when  I 
tried  to  explore  that  with  her 
there  wasn't  much  recognition/or  it 
didn't  seem  to  be  a problem.  He's 
been  a concerned  classmate/advocate 
for  her.  I wrote  "I  know/ I don't 
know"  on  the  board  and  we  said  it 
in  the  various  languages . 

Phuth  was  copying  Y's  homework. 
(He  gave  it  to  me  again  today.  He 
asked,  as  always,  when  I looked  at 
it,  "Wrong,  right?") 

Everyone  but  Phuth  had  done  the 
homework  and  P and  Panna  didn't 
have  it  with  them — don't  know  if 
they'd  done  it  at  home  or  not. 

This  Thursday's  homework  is  all 
Yang's  idea,  and  it  makes  for  a 
neat  (i.e.  self-contained)  and 
pleasant  Monday.  Maybe  starting 
all  together  and  letting  Tues/Wed 
be  a little  more  individualized  is 
a way  to  go? 


Excerpts  from  the  author's  classroom  log 
are  included  throughout  the  article. 
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misread  an  entire  line  out  of  sequence.  In  order  to 
work  from  this  strength  (her  ability  to  memorize), 
lasked  her  to  tell  me  the  words  she  most  wanted  to 
be  able  to  read  and  write  and  wrote  those  words  in 
her  notebook.  Her  list  included:  church,  work, 
husband,  children,  grandchildren  and  teacher. 
Whenever  these  words  appeared,  I called  her  atten- 
tion to  them,  hoping  to  help  her  develop  a sight 
word  vocabulary  and,  later,  to  point  out  the 
graphaphonic  cues  she  could  use.  Although  she 
asked  for  the  highlighters  when  I forgot  to  bring 
them  to  class,  it  wasn’t  until  I tried  making  flash- 
cards  for  her  that  she  really  responded. 

Rosalie  is  somewhat  atypical  as  an  ESL/literacy 
learner,  because  her  aural/oral  abilities  are  so 
strong.  She  was  able  to  tell  me  that  the  flashcards 
were  the  best  thing  for  her  and  that  she  was  making 
sentences  in  school  and  at  home  to  study,  copy,  and 
learn.  She  is  also  able  to  provide  very  important 
feedback.  When  another  learner  helped  her,  she 
became  confused,  but  politely  accepted  his  assis- 
tance. When  he  left  the  room,  she  told  me,  “Joe’s 
help  is  no  good.”  She  asked  other  learners  in  the 
class  for  assistance,  but  she  knew  that  Joe  moves  too 
quickly  and  too  assertively  for  her.  Rosalie  can 
report  on  her  own  learning  in  a way  that  other  ESL 
learners  cannot  often  do.  Although  her  ability  to 
speak  enables  us  to  hasten  our  process,  similar 
strides  can  be  made  with  other  learners. 

Bosilio,  a Puerto  Rican  learner  who  emigrated 
to  the  mainland  in  1973,  presented  himself  to  the 
Literacy/ESL  Program  with  the  goal  of  learning  to 
read  in  Spanish  so  that  he  could  read  the  Bible. 
Bosilio  was  the  only  student  to  arrive  during  the  first 
week  of  morning  classes,  so  he  was  essentially  in  a 
private  tutorial.  I began  working  with  very  simple 
LEA  sentences  (not  stories)  in  order  to  enable  him 
to  learn  sight  words  like  “name”  and  “nombre.” 
However,  during  the  first  two  and  a half  hour 


session,  he  made  it  clear  that  he  wanted  to  know  the 
alphabet  because  that  was  the  only  way  he  would 
learn  to  read.  My  explanation  that  learning  the 
sounds  of  the  letters  might  be  more  important  than 
learning  their  names  was  not  convincing.  So, 
during  the  second  session,  we  worked  hard  on 
drilling  vowel  sounds  (a,e,i,o,u)  in  Spanish  and 
then  working  with  syllables  (e.g.,  ma,  me,  mi,  mo, 
mu).  This  appealed  to  Bosilio,  and  he  copied  pages 
of  similar  syllabic  drills  into  his  notebook  between 
classes.  I tried  several  times  to  introduce  some 
meaning,  by  showing  him  “so”  and  “pa”  from 
his  lists,  and  writing  sentences  like,  “Me  gusta  la 
sopa.”  When  he  heard  the  sound  “si”  he  recog- 
nized it  as  the  “si”  in  Bosilio.  When  he  saw  “si” 
out  of  the  context  of  a horizontal  list  or  in  the  context 
of  a sentence,  e.g.,  “Si,  me  gusta  la  sopa,  he  was 
unable  to  recognize  it.  If  he^was  reading  through 
the  vowel/consonant  lists  and  came  to,  “so”  and 
was  unable  to  recognize  it,  he  would  go  back  to  the 
top  of  his  list  (sa,  se,  si,  so,  su)  and  recite  it  until 
he  came  to  the  sound  in  question. 

Bosilio  was  able  to  make  connections  on  the 
basis  of  sound  more  than  he  was  able  to  relate  to 
visual  cues.  In  the  10  hours  we  spent  together 
during  our  first  week,  he  memorized  the  vowels  (in 
sequence)  but  was  able  to  retain  few  whole/ sight 
words.  The  following  week,  nine  new  students 
entered  the  class,  five  Russians  and  three  Hispanics. 
Two  of  the  Hispanic  learners  were  inappropriately 
placed  in  the  class;  their  language  and  literacy 
abilities  far  outstripped  the  others.  Both  learners 
were  willing  to  stay  in  the  class  until  space  in  a more 
advanced  class  became  available.  Both  also  agreed 
to  work  with  Bosilio  on  his  Spanish  reading  and 
writing.  By  the  end  of  the  week,  however,  Bosilio 
had  decided  to  leave  the  class,  stating  that,  although 
he  received  some  help  from  the  other  learners,  he 
did  not  want  to  participate  in  any  of  the  whole  class 
English  language  reading,  writing,  or  discussion. 
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Additionally,  he  was  convinced  that  he  would  only 
learn  by  learning  to  read  Spanish.  He  was  absent 
the  following  day;  I contacted  him  with  information 
about  a Spanish  language  literacy  class  being  held 
on  Saturdays.  He  thanked  me  for  the  information 
and  did  not  return  to  the  Institute. 

Bosilio’s  story  challenges  assumptions  about 
learners  who  won’t  or  can’t  explicitly  express  their 
learning  goals.  Bosilio  wanted  to  learn  to  read  the 
Bible.  His  life  in  the  church  was  of  paramount 
importance  to  him.  He  reported  that  the  church  and 
Christ  had  saved  him  from  a life  of  alcoholism  and 
womanizing.  Clearly,  reading  the  Word  was  a 
dominant  motivation.  Bosilio  was  receiving  some 
sort  of  disability  assistance  and , although  he  did  not 
discuss  the  nature  of  his  disability,  there  was 
something  not  quite  right  affectively  in  his  interac- 
tion with  me  and  with  others.  (His  interpersonal 
interactions,  his  sense  of  personal  “space”  be- 
tween himself  and  his  interlocutors,  and  his  speech 
rhythms  indicated  behavioral  differences  between 
Bosilio  and  other  people. ) His  needs  exceeded  mere 
literacy  learning  (as  do  many  learner’s).  However, 
his  intractability  vis  a vis  learning  even  some  of  the 
time  with  people  speaking  English  indicated  to  me 
that  no  amount  of  cajoling  would  or  necessarily 
should  have  persuaded  him  to  contemplate  a more 
open  attitude  toward  learning.  In  his  case,  owner- 
ship extended  beyond  actual  literacy,  per  se,  to  the 
ways  in  which  he  felt  he  could  become  literate. 
Assessment  moves  beyond  noting  his  ability  to  copy 
lists  of  phonetic  drills  and  into  observation  of  and 
reflection  upon  his  interactions  with  other  learners, 
his  motivation  for  learning,  and  the  goals  which 
drove  him  to  seek  classes  in  the  first  place. 

Marines  said  she  wanted  to  take  her  GED  and 
was  fairly  fluent  orally.  She  wanted  to  pass  a 
licensing  test  in  English  to  work  as  a beautician,  as 
she  had  done  in  Puerto  Rico.  She  was  able  to 


Tuesday,  November  5 

15  students/R's  daughter 

D the  volunteer  came  to  visit  and 
became  a language  experience. 

I started  two  sentences,  asked  W 
and  Y to  write,  and  then  they 
asked  him  questions  as  well. 
Everyone  read. 

After  the  break,  we  finished 

wuc  a iviugnv4i\<  iiw  w j — 

one  read  individually,  so  that  we 
could  get  to  talking  abput  why 
people  steal — because  they  need 
money;  need  money  to  buy  drugs, 
are  lazy  or  laid  off,  or  because 
they're  poor  or  bad.  I kept 
asking  why  and  as  we  only  had 
about  10  minutes,  asked  if  they 
wanted  to  discuss  the  question 
tomorrow,  which  they  do. 

Y tried  to  cash  a check  with 
Audry;  I'll  deposit  it  for  her 
and  hope  it's  OK . 

E came  late,  went  to  the  b/b  to 
read  anyway. 

P said  she'll  be  out  Wednesday 
and  Thursday,  because  of  the 
wedding . 

R's  still  tired.  P and  L were 
early. 
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communicate  adequately  in  writing,  yet  she  con- 
tinually made  sloppy  mistakes— the  kinds  of  errors 
that  I might  point  out  to  a learner  as  they  occur 
repeatedly  over  time  (e.g.,  spelling  errors  in  high 
frequency  words,  misuse  of  past  tense).  Because 
she  explicitly  stated  that  she  wants  to  pass  a product 
oriented  test,  I adapted  my  style  to  try  to  accommo- 
date her  needs.  I told  her  that  she  knows  a lot  and 
can  write  pretty  well,  but  that  the  test  looks  for 
correct  grammar,  particularly  agreement  in  verb 
tenses,  and  the  ever-troublesome  correct  usage  of 
“have”  and  “has.”  Nonetheless,  she  repeatedly 
worked  too  quickly  and  made  the  same  mistakes  she 
had  no  doubt  made  for  years  before  coming  back  to 
school. 

I noticed  that  Marines  was  endlessly  patient 
helping  other  learners  and  had  real  strengths  as  a 
reader.  She  enjoyed  reading  passages  from  stan- 
dard ESL  reading  texts  and  easily  completed  the 
comprehension  tests  at  the  end  units.  How  could  I 
help  her  pass  the  writing  sample  of  the  GED?  I 
needed  to  continue  to  ask  her  about  the  reading  and 
writing  she  does  outside  of  school  and  perhaps  to 
further  break  down  the  components  of  her  writing 
that  are  problematic--“Let’s  look  at  all  the  verbs 
in  this  paragraph;  tomorrow  we’ll  talk  about  the 
pronouns.”  Again,  assessment  was  ongoing  and 
fluid,  and  based  around  the  need  expressed  by  this 
learner. 

Phuth  has  been  coming  to  class  for  years. 
Initially  able  to  only  copy  and  repeat,  she  developed 
a sight  word  vocabulary  that  enables  her  to  respond 
to  most  dialogue  journal  entries  with  little  assis- 
tance from  others.  She  can  generate  sentences  at  the 
blackboard  and  can  help  others  at  the  board  with 
spelling  questions.  She  decodes  high  frequency 
words  with  ease,  yet,  she  still  needs  to  develop 
strategies  which  will  enable  her  to  use  context  cues 
more  effectively.  If  she’s  reading  the  word  “clean” 


by  itself,  she’ll  say,  “I  forgot,  teacher”,  or  “I 
don’t  know.  ” Yet,  if  she  wants  to  tell  me  that  she 
cleaned  her  house  last  weekend,  she’ll  look  in  her 
notebook  and  find  the  word  there  so  that  she  can 
copy  it  into  her  journal.  Although  she  has  made 
great  strides  with  print,  she  still  miscopies  entire 
lines  of  text.  Nonetheless,  she  now  provides  help 
to  other  learners  in  addition  to  receiving  it  herself. 
She  can  speak  Lao  and  Cambodian  and  helps  other 
learners  who  may  not  understand  a discussion  in 
English.  She  still  copies  everything  that’s  written 
on  the  blackboard.  Her  grown  children  help  her  at 
home.  Her  physical  problems  (chronic  back  pain) 
make  employment  impossible.  For  Phuth ’s  needs- 
-to  be  among  people,  to  have  some  independence 
with  print  and  with  language— the  class  seems 
adequate. 

Luis  passed  his  citizenship  test  and  yet  realized 
that  he  has  to  continue  to  study  to  become  more 
fluent  as  a reader  and  writer.  His  native  Portuguese 
affects  his  hearing  and  encoding  of  words;  he  writes 
words  as  he  hears  and  pronounces  them.  “I  luk  a 
stoa,”  means  I look  at  the  stores.  The  letters  r and 
h in  Portuguese  can  be  interchanged  in  certain 
instances,  so  Luis  will  occasionally  “rave  a good 
weekend.  ” As  I listen  to  his  speech,  read  his  words, 
and  learn  about  his  language,  I continually  adjust 
the  guidance  I try  to  give  him  so  that  his  reading  and 
writing  in  English  gradually  become  more  fluent. 

These  observations  about  learners’  daily  inter- 
action with  print  and  language  have  accrued  over 
time.  I document  what  transpires  in  each  class 
shortly  after  the  class  period,  noting  who  asks 
whom  for  help,  what  materials  someone  might  have 
brought  from  home,  what  someone  was  interested 
in  reading  and/or  writing  during  that  class  period, 
who  volunteered  to  go  to  the  blackboard  to  write, 
who  asked  for  more  homework,  etc.  I make  notes 
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of  miscues  or  questions  directly  on  my  copy  of  a 
particular  reading  passage,  or  in  a notebook  I keep 
on  hand.  Over  time,  when  reviewing  the  notes,  I 
can  see  what  day  it  was  that  Rosalie  told  me  she  liked 
flashcards  and  wanted  more,  when  Tomas  decided 
to  bring  in  Christmas  cards  from  home  for  us  to 
write  to  people  in  the  hospital,  when  it  was  that  Joe 
was  over  zealous  with  Rosalie,  when  it  was  that 
Teresa  read  through  an  entire  passage  without 
asking  for  assistance,  when  it  was  that  Luis  began 
to  realize  that  r and  h are  not  interchangeable. 

I also  see  more  subtle  patterns  emerge  and  my 
own  assumptions  challenged.  I was  told  that  Alice 
was  illiterate  and,  because  she  arrived  during  the 
middle  of  a class  session,  was  unable  to  do  more 
than  welcome  her  and  invite  her  to  copy  the  passage 
that  we  were  working  on . Although  she  spoke  very 
little  English,  she  was  able  to  decode  the  story,  and 
I learned  that  she  was  literate  in  French,  and  was 
therefore  most  likely  able  to  make  fairly  rapid 
progress,  transferring  her  knowledge  of  reading 
from  one  language  to  another.  She  also  brought  two 
large  Bibles,  written  in  French,  giving  me  a sense 
of  another  way  in  which  literacy  figures  in  her  life. 
Her  subsequent  questions,  in  French,  and  gradually 
in  English  about  my  own  religious  beliefs,  and  her 
descriptions  of  activities  at  her  church,  confirmed 
my  suspicion  that  she  is  indeed  quite  literate  in 
French  and  needs  to  gain  literacy  in  English  in  order 
to  find  a job. 

Yvette,  also  from  Haiti,  wants  me  to  sit  with  her 
and  point  out  words  on  her  paper  as  others  read 
aloud.  She  has  virtually  no  retention  of  sounds, 
letters  or  words,  and  yet  complains  if  Alice  doesn’t 
speak  English  (Yvette  speaks  fairly  well,  if  with 
some  difficulty).  She  comes  to  class  regulary  but 
has  a sister  do  her  homework  for  her.  She  appears 
to  be  motivated  to  learn.  Yet,  even  when  I sit  with 
her  for  an  extended  period  of  time,  her  attention 
wanders  from  the  paper.  She  likes  to  dictate  brief 


Wednesday , November  6 
11  students /R's  daughter 

We  began  with  a small  group,  so 
I decided  to  give  them  their 
journals  first,  which  took  us 
through  the  first  hour.  K 
didn't  want  to  write  in  a jour- 
nal. R said  that  he  had  a lot 
of  homework.  He  did  a beautiful 
drawing  as  well,  but  he  kept  it. 

Everyone  was  pretty  well  engaged 
with  the  journals.  Y copied  my 
questions  to  her — even  after  M 
had  read  the  questions  to  her — 
doing  her  book  before  his  own. 

I read  them  to  her,  solicited 
her  answers  and  wrote  her  re- 
sponses in  her  notebook  so  that 
she  could  copy  them  into  her 
bluebook  later. 

L was  intensely  absorbed  in  the 
journal  and  asked  for  virtually 
no  help.  P came  in  late,  and 
unseen,  and  only  after  someone 
pointed  out  her  presence  did  I 
see  her,  give  her  her  book. 

After  the  break,  we  read  the 
write  up  of  DJ's  visit  and 
started  talking  about  the  ques- 
tions, which  we'll  finish  tomor- 
row. W Y actually  completed  the 
questions;  not  too  many  people 
had  gotten  to  writing  them  yet. 
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stories  to  me  which  she  then  copies  into  her 
notebook.  The  notebook  has  entries  that  begin  at 
the  top  of  a page  and  other  entries  that  begin  at  the 
bottom  of  a page.  She  doesn’t  appear  to  know  that 
print  and  books  go  from  left  to  right.  Yet,  unless 
she’s  working  overtime,  she’s  in  class  early  every 
night. 

Although  she  constantly  says  that  she  can’t 
read,  other  learners  always  include  her  when  they 
go  to  the  board  to  read.  (One  learner  reads  and  then 
asks  another,  etc.)  The  other  learners  are  patient 
and  help  her  as  she  “reads”  what  she  remembers 
and  what  they  tell  her  the  words  say.  Her  progress 
will  be  very  slow.  As  I leant  more  about  the  kinds 
of  literacy  activities  that  appeal  to  her,  I hope  to  be 
able  to  find  better  ways  to  facilitate  that  progress. 

What  Does  It  Mean? 

The  point  of  sharing  these  observations  and 
vignettes  is  to  show  that  ongoing  assessment  is 
exactly  what  it  claims  to  be— a process  of  observa- 
tion, reflection,  and  feedback  aimed  at  assisting 
learners* 1 2 3 4  and  facilitators.  Through  systematic 
documentation  of  classroom  activity  and  subse- 
quent reflection  about  learner  progress,  as  well  as 
explicit  talk  with  learners,  I try  to  continually  adjust 
and  individualize  the  facilitating  work  I do  in  the 
classroom  so  that  learners  can  find  better  and  more 
appropriate  means  of  learning.  Periodic  pencil/ 
paper  surveys  are  part  of  this  process,  (appendix  4) 
as  are  explicit  questions  asked  now  and  again  of 
particular  learners:  “Do  you  like  working  together 
with  other  people?  Did  you  like  writing  about  the 
photographs?  What  would  you  like  to  work  on 
today?  Why  do  you  think  this  is  easy  and  that  is 
difficult?” 

Finally,  ongoing  assessment  is  a part  of  the 
learning  process.  The  forms  that  assessment  re- 
ports may  take-in-house  files,  funders’  reports, 
reports  written  for,  by,  or  with  learners- will  de- 


pend on  the  reasons  for  which  they  are  written.  For 
myself,  as  a practitioner,  the  most  important  piece 
is  the  process  itself— the  ongoing  learning,  rethink- 
ing, reflection  needed  for  facilitating  learning  for 
others  and  for  myself. 

Janet  Isserlis  teaches  ESL/Literacy  at  the  Interna- 
tional Institute  of  Rhode  Island. 


Notes 

1 . Action  research  is  a process  through  which  one  systemati- 

cally examines  what  happens  in  the  classroom,  reflects 
upon  activity  and  subsequently  modifies  that  activity  in 
order  to  improve  teaching  practice.  In  a larger  context, 
action  research  describes  collaborative  investigative  pro- 
cesses through  which  to  enact  social  change. 

2.  For  more  on  documenting  progress  see  Lenore  Balliro, 

Reassessing  Assessment  in  Adult  ESL/Literacy , (Paper 
presented  at  TESOL,  San  Antonio,  1989),  p.  35.  The 
paper  provides  an  excellent  overview  of  assessment  and 
is  available  through  ERIC. 

3.  Collignon,  Isserlis,  and  Smith.  A Handbook  for  Practitio- 

ners: ESL/Literacy  for  Adult  Non-native  Speakers  of 
English . (Providence,  RI:  International  Institute  of 
Rhode  Island,  1990).  See  also,  Isserlis.  "Using  Action 
Research  for  ESL  Literacy  Evaluation  and  Assessment. " 
TESL  Talk , Vol.  20,  No.  1 (1990). 

4.  See  also,  Don  Robishaw’s  remarks  in  Adventures  in 
Assessment,  Vol.  1,  p.37  (1991). 
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Three  by  Three  by  Four: 
Ongoing  Assessment  at  the 
Community  Learning  Center 


Karen  Ebbit,  Priscilla  Lee,  Pam  Nelson,  and  Joann  Wheeler 


. • V he  Community  Learning  Center/SC ALE 
lllll  Assessment  Project  began  when  a group  of 
ESL  teachers  in  Cambridge  and  Somerviiie 
realized  they  shared  an  interest  in  exploring  alter- 
native methods  of  assessment.  This  article  docu- 
ments the  process  followed  by  the  assessment  team 
at  the  Community  Learning  Center:  Karen  Ebbitt, 
Priscilla  Lee,  Pam  Nelson,  and  Joann  Wheeler. 
Priscilla,  Pam,  and  Joann  conducted  the  field  tests 
with  their  ESL  I/literacy  classes.  Karen,  who  was 
not  teaching  a level  I/literacy  class  during  the 
project,  did  not  participate  in  field  testing  but 
participated  in  the  planning,  development,  and 
revision  of  the  tools  and  process. 

Assessment  at  CLC 

At  CLC,  assessment  had  largely  been  a matter 
ofindividual  teacher’s  choice.  Some  tested  weekly; 
some  did  teacher-made  tests  at  the  end  of  a cycle; 
and  others  did  not  test  formally,  but  assessed 
progress  on  the  basis  of  classroom  performance. 
The  ESL  department  had  discussed  assessment  as 
something  the  students  valued  but  had  not  devel- 
oped a consistent  overall  process.  In  working  with 
SCALE,  CLC  staff  hoped  to  create  a process  which 
would  be  comfortable  and  effective  for  students  and 
teachers,  and  which  would  also  satisfy  reporting 
requirements  for  various  funding  sources. 

The  teachers  began  a series  of  four  two-hour 


meetings  at  which  project  staff  from  both  centers  set 
general  goals  and  examined  options  for  alternative 
assessment.  The  group  first  looked  at  the  relation- 
ship between  assessment,  goal-setting,  and  curricu- 
lum and  then  reviewed  various  tools  for  alternative 
assessment  options  including  tools  developed  by 
other  Massachusetts  programs. 

The  teachers  decided  in  the  first  goal-setting 
meeting  to  limit  the  scope  of  the  project  to  develop- 
ing an  assessment  tool  which  could  be  used  at  the 
ESL  I/ESL  literacy  level.  Their  rationale  was  that 
if  a tool  was  usable  at  a minimal  language  level,  it 
could  easily  be  adapted  for  use  with  students  who 
possessed  more  advanced  skills.  The  overall  goal 
was  to  address  not  only  the  assessment  of  language 
skills  acquired  in  a given  period  of  time,  but  also 
that  of  learning  strategies  and  study  skills. 

A Three-Part  Process 

After  the  four  research  meetings,  project  staff 
met  separately  at  each  center  to  design  assessment 
tools  that  would  suit  the  needs  of  their  staff  and 
students.  The  Community  Learning  Center  created 
a three-part  process  consisting  of  a goal-setting 
exercise,  a weekly  self-asSessment,  and  a progress 
record.  The  assessment  tools  were  field  tested  in 
three  ESL  classes  during  two  five-week  summer 
sessions  in  July  and  August,  1991.  Accounts  of 
these  field  tests  by  the  three  teachers  who  used  the 
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process  in  their  classes  appear  below.  The  forms, 
both  current  and  revised,  appear  also. 

Part  1 . The  Goal-Setting  Exercise 

In  the  initial  goal-setting  exercise,  students 
prioritize,  in  pairs  or  as  a group,  the  language 
survival  areas  they  would  like  to  explore  in  their 
ESL  class.  Tools  for  this  exercise  consist  of  five  8 
1/2"  x 1 1 " drawings  of  survival  areas  and  one  sheet 
on  which  all  the  drawings  are  reduced  so  teachers 
can  copy  and  cut  them  to  make  decks  of  cards. 
(figures  1 and  2,  Appendix  5)  The  pictures  provide 
a context  in  which  students  can  discuss  where  they 
need  to  use  English  and  which  survival  areas  they 
think  are  most  important  for  them  to  learn.  A box 
is  drawn  on  each  card  for  students  or  pairs  to  write 
in  a priority  number— one  (for  “very  important”) 
through  five  (for  “not  so  important”). 

Teacher  A:  Goal-Setting  Exercise 

Objective:  To  help  students  think  about  situations 
where  they  need  to  use  English  and  choose  which 
area  they  want  to  work  on. 

I began  the  goal-setting  exercise  by  posting  the 
large  topic  pictures  and  giving  the  students  a chance 
to  read  the  captions  and  talk  about  the  pictures. 
Then  we  discussed  where  English  was  necessary. 
Many  of  the  students  agreed  that  they  did  not  need 
to  speak  much  English  to  go  shopping;  some  said 
that  English  was  necessary  at  their  workplace; 
others  were  not  working.  Of  the  three  remaining 
cards,  we  talked  about  whether  using  English  on  the 
telephone,  in  the  doctor’s  office,  or  in  conversation 
with  classmates  was  easy  or  difficult. 

I passed  out  the  sets  of  small  topic  cards  and 
asked  the  students  to  prioritize  the  topics,  using  one 
for  “very  necessary”  through  five  for  “easy  for 
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me.  ” The  pairs  got  to  work.  I thought  the  activity 
was  going  well  when  I realized  that  one  man  had 
instructed  everyone  in  Spanish  to  fill  out  their  cards 
in  a particular  way— and  he  had  completely  misun- 
derstood the  exercise!  He  thought  that  I wanted  to 
know  whether  he  knew  the  Spanish  translation  for 
the  captions  on  the  cards,  so  he  labelled  the  cog- 
nates, like  “telephone”  and  “doctor,”  as  easy, 
and  marked  the  others  as  difficult.  Everyone  had 
followed  his  lead,  so  I decided  to  abandon  the  pair 
work  and  return  to  the  large  pictures  on  the  board. 

We  repeated  the  discussion  of  situations  where 
English  is  difficult  and  then  voted  asa  group  on  each 
picture.  The  doctor’s  office  and  the  telephone  got 
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an  equal  number  of  votes  for  difficulty,  so  that  was 
what  we  determined  to  work  on  for  the  five  weeks 
of  the  class. 

This  goal-setting  exercise  took  20  minutes. 

Teacher  B:  Goal-Setting  Exercise 

Objective:  To  prioritize  from  a list  the  topics 
students  want  to  study. 

I began  the  goal-setting  exercise  by  writing  on 
the  board:  “What  do  I want  to  study?”  After  the 
students  read  the  question  to  themselves  and  aloud, 
I began  to  go  through  the  large  topic  cards  with  the 
class.  We  discussed  them  in  turn,  identifying  the 
items  in  each  picture.  I allowed  time  for  students  to 
say  whatever  they  wanted  about  a particular  topic 
before  moving  on. 

When  we  had  gone  through  the  topics,  I turned 
to  the  board  and  read  the  question  again . I spread  all 
the  topic  cards  along  the  blackboard  ledge  and  had 
one  student  ask  me  the  question.  I modeled  how  to 
prioritize  what  I wanted  to  study  by  writing  the  five 
topics  on  the  board  and  putting  them  in  order  of 
preference.  I explained  that  "one"  designated  the 
topic  they  most  wanted  to  study  and  "five"  what 
they  were  least  interested  in.  Students  paired  and 
took  their  own  set  of  topic  cards.  I asked  the  pairs 
to  choose  what  they  wanted  to  study  and  write  it  on 
a separate  piece  of  paper. 

I walked  around  the  room  during  this  part  of  the 
procedure  and  talked  with  students,  hoping  to  get  an 
idea  whether  they  understood  the  task.  The  13 
students  came  up  with  the  following  “votes.” 

Shopping  415541554554113 

Jobs  121111111312233 

Doctor  332233222225442 


POOD 


shopping- 

P 


FIGURE  2 


Telephone  243322333  3 1 3123 

How  are 

you  feeling?  544544454445554 

I assumed  my  explanation  of  how  to  prioritize 
was  understood,  but  thought  differently  when  I 
reviewed  the  students’  lists.  I saw  that  they  had 
copied  the  order  I had  written  on  the  board  even 
though  I had  erased  my  numbers  before  the  students 
chose  their  own.  I had  left  the  list  of  five  topics  on 
the  board  and  the  students  seemed  to  adhere  to  the 
order  of  that  list. 

Nevertheless,  I decided  that  whatever  we  worked 
on  would  be  useful,  so  we  decided  on  the  topic  of 
jobs.  I talked  with  the  class  before  figuring  out  how 
to  teach  this  topic  because  I wanted  to  get  a feel  for 
what  the  students  wanted.  The  class  was  composed 
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IW  CLASS 

1*  Ho**  I knov  thaaa  naw  worda:  ^ 1 £*  . ^ CLL  . 

‘f' ii. » StVOl  fen', 
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3-  I don’t  undaratand 


*.  In  clMa  tbla  waak  I Ukad  1 Ilk*  d £ t^6  KY  Ah t HQ  , 


5.  In  claa*  thla  waak  I didn’t  Ilka 


6.  In  elaaa  I lUtanad  to  ^V^fbnA^'  ; w.fc  tajliah. 

7 . la  elaaa  I apokt  Entllrt  to  Z 3p3  ke  "tO  rr.i/  r 1 0~S  S ni  cW  ?£  . 

8.  In  elaaa  I read  yg<ld  £,  bpt/f  bo<jf^  *A<I 

».  In  elaaa  I vreta  & bO\)t  ^ In  inxllah. 


10.  In  elaaa  I saad  help  with 

S $f>ej<\ry  w 


± Hie  A help  leiLth  p^onDnufion  . 


OUTSIDE  CLASS 

1.  IMa  weak  I apoka  Ea*llah  to  mw  fc.  ml  fo  M*  %\)PtF  VhS 

2.  Thla  weak  I llatenad  to  5 f~0f»  n /■■  A <l*V>g  apeak  En*llah. 

3.  Thla  weak  I read  ? K»»*A  £ if  < H<t*)fP.ln  Ea,li,h. 

4.  Thla  weak  I wrote  fl.’f  hnn  U';,  0*  I j jj  rW  1/  t-  fa^iiah. 


FIGURE  3 


want  to  study.”  I asked  the  learners  to  spread  out 
the  topic  cards  and  order  them  in  answer  to  the 
question:  “What  is  important  for  you  to  learn  in 
English?”  I modeled  this  exercise  for  the  students. 

The  students  were  paired  and  I asked  them  to 
decide,  with  their  partners,  on  the  sequence  of  the 
cards,  substituting  their  preference  for  those  just 
modelled  by  me. 

The  students’  cooperative  work  took  about  five 
minutes . While  the  pairs  conferred , I walked  around 
the  room  and  observed  that  the  students  didn’t  have 
problems  interacting  in  English  with  each  other  or 
agreeing  on  how  to  prioritize  the  cards. 

The  topic  cards  were  numbered  by  the  pairs  of 
learners,  with  one  signifying  the  topic  as  “most 
important  for  us  to  learn”  and  two  through  five 
designating  the  remaining  topics  in  descending 
order  of  importance.  This  is  how  the  five  pairs  of 
learners  sequenced  the  topics: 

Doctor  2 2 1 0 0 

Shopping  2 12  0 0 


mainly  of  women  who  either  worked  in  the  home  or 
worked  elsewhere  as  cleaning  women  in  nursing 
homes,  offices,  or  universities.  I decided  to  focus 
our  attention  on  vocabulary  having  to  do  with  tasks/ 
jobs/chores.  My  decision  to  limit  the  scope  of  our 
work  was  also  based  on  the  fact  that  our  class  met 
only  twice  a week  for  five  weeks. 

Teacher  C:  Goal-Setting  Exercise 

Objective:  To  implement  a quick  prioritizing  exer- 
cise so  learners  can  direct  the  topical  content  of 
their  course. 


Jobs/Work 


2 0 


Telephone  0 0 


3 1 


How  are 

you  feeling?  0 10  0 4 


The  prioritizing  tool  was  easy  to  use  and  helped 
the  students  collectively  choose  the  topical  content 
of  their  ESL  class. 


Part  2.  The  Weekly  Self-Assessment 


I began  by  explaining  the  purpose  of  the  goal- 
setting activity:  “This  will  help  me  know  what  you 


The  weekly  self-assessment  is  a one  page  record 
of  new  words,  grammar,  etc.  learned  in  class  and 
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outside  of  class,  intended  to  increase  the  students’ 
perception  of  what  they  learned,  how  they  learned, 
and  where  they  learned. 

The  original  form  (figure  3,  appendix  6)  was 
too  difficult  for  the  students  to  use  so  it  was 
shortened  and  simplified,  (figure  4,  appendix  7)  We 
wanted  a simpler  format,  one  which  left  out  ques- 
tions we  had  judged  uninformative,  and  one  which 
appeared  less  intimidating  to  learners  accustomed 
to  short  sentences.  I tried  out  a shorter  form-pared 
down  from  14  to  six  questions.  This  revised  form 
retained  questions  which  had  previously  been  an- 
swered with  relative  ease  and  which  had  been 
helpful  for  the  instructor  and,  apparently  for  the 
students.  Using  the  form  in  class,  the  teachers  found 
it  useful  to  go  through  the  revised  form  as  a group 
for  the  first  few  weeks,  having  the  class  brainstorm 
new  words  or  possible  responses.  After  it  became 
routine,  students  were  able  to  complete  the  form  by 
themselves. 

Teacher  A:  Weekly  Self-Assessment 

Objective:  To  give  students  an  opportunity  to  reflect 
on  the  learning  process  in  and  outside  of  class,  and 
to  provide  the  teacher  with  feedback  on  what  the 
students  think  they  are  learning. 

I decided  to  use  the  revised  and  simplified  form 
which  had  been  developed  during  July.  When  we 
had  completed  one  full  week  of  classes,  I introduced 
the  self-assessment.  I explained  that  their  answers 
would  help  me  know  what  they  had  learned.  I drew 
a large  version  of  the  form  on  the  board. 

Everyone  filled  in  their  name  and  the  date 
without  problems.  I had  anticipated  trouble  with 
filling  in  “#  of  classes  last  week,”  so  I asked  the 
students  to  write  how  many  classes  they  had  at- 
tended so  far.  In  two  cases,  students  claimed  to  have 
been  in  class  when  they  weren’t- maybe  because  of 
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the  school’s  strict  attendance  policy;  but,  I decided 
not  to  make  an  issue  of  it.  We  read  the  first  question 
together  from  the  board,  and  I clarified  by  saying, 
“Before,  no,”  “Now,  yes.”  At  first,  no  one  was 
willing  to  admit  to  understanding  anything— either 
"before"  or  "now.  "Sol  prompted  them  with  some 
of  the  things  we  had  worked  on,  such  as  body  part 
words  and  remedies.  They  agreed  that  they  under- 
stood those  words,  and  some  students  wrote  a few 
vocabulary  words  in  the  space  provided. 

I had  left  half  an  hour  at  the  end  of  the  class  to 
do  this  assessment,  and  we  ran  out  of  time  after 
completing  the  first  two  statements.  I asked  the 
students  to  take  the  forms  home  with  them  and  bring 
them  back  to  class  next  time.  The  hurricane  inter- 
vened, however,  and  I decided  not  to  return  to  this 
exercise. 


• - 

' Nl  AM  E/p^tE  =#.  of  classes 

l 05+  . 


1.  New  I TcJc  L . 
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3.  Thl*  week  I apokt  English  
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Observations:  I think  the  revised  form  could  be 
used  without  difficulty  at  higher  levels,  but  my 
students  and  I found  it  a frustrating  exercise. 

Teacher  B:  Weekly  Self-Assessment 

After  we  had  met  for  four  classes,  I handed  out 
the  revised  self-assessment  form.  I was  mainly 
interested  in  how  the  students  answered  the  first 
question.  I explained  that  the  form  would  help  me 
know  what  they  had  learned  during  our  first  four 
meetings.  Then  I handed  out  the  form  and  drew  a 
version  of  it  on  the  board. 

The  students  did  not  have  any  trouble  filling  in 
name  or  date.  However,  there  was  some  confusion 
about  the  “#  of  classes  last  week”  because  of 
classes  missed  during  the  hurricane  and  because  the 
class  met  only  twice  each  week.  To  simplify  the 
process,  I decided  to  go  through  my  attendance 
sheet  and  tell  the  students  how  many  days  to  write 
down. 

I wrote  the  first  open-ended  statement  on  the 
board  and  asked  students  to  read  it.  I explained  the 
statement  and  asked  students  to  list  those  words 
which  they  now  understood  because  of  our  lessons. 
We  wrote  a list  of  those  new  words  on  the  board  and 
students  chose  from  this  list  for  their  own  form.  We 
did  not  go  further  with  this  form  than  the  first 
question. 

Observations:  The  weekly  self-assessment  form 
proved  problematic  for  my  class.  The  concept  of 
“now  I understand”  seemed  strange  to  them  and 
I wondered,  after  the  class,  whether  they  would 
have  preferred  “now  I know.”  Perhaps  the  idea  of 
writing  a list  of  words  was  confusing.  Whatever  the 
cause,  this  activity  left  my  class  scratching  their 
heads.  I continued  to  scratch  my  own  head,  wonder- 
ing why  the  questionnaire  had  failed.  Initially  I 
thought  that  I would  bring  back  the  form  in  a new 
version  for  the  next  class.  However,  I decided 


against  it  due  to  the  very  limited  number  of  class 
hours  we  had;  filling  out  the  form  took  too  much 
time.  I believed  that  the  progress  record  would 
provide  the  information  we  were  looking  for. 

Teacher  C:  Weekly  Self-Assessment 

I distributed  the  original  weekly  self-assess- 
ment form  to  my  students  after  six  hours  of  class- 
room activity  centered  around  “doctor,”  their 
most  preferred  topic.  My  brief  introduction  was, 
“This  will  help  me  to  know  what  you  learned  last 
week  and  it  will  help  me  to  teach  you  better.” 
Students  filled  out  the  form  as  a group.  We  used  the 
blackboard  to  record  some  of  the  responses  that 
came  out  of  a collective  interpretation  of  the  open- 
ended  statements  under  “in  class”  and  “outside 
class.” 

Students  wrote  in  their  names  with  no  trouble. 
For  the  week  ending,  I asked  the  students  to  write 
in  “July  7, 199 1 , ” indicating  the  Friday  of  the  past 
week.  This  caused  some  initial  confusion  because 
some  of  the  students  anticipated  writing  the  current 
date  in  the  space,  and  two  others  had  already  done 
so.  Once  I explained  that  the  date  was  to  signify 
“the  end  of  last  week,”  students  were  further 
confused,  pointing  out  that  weeks  end  on  Saturdays! 

“How  many  hours  in  class  this  week?”  was 
filled  in  with  some  hesitation.  If  a student  left  class 
10  minutes  early  on  Tuesday,  did  that  need  to  be 
accounted  for  in  the  total?  I felt  that  some  students 
were  reluctant  to  record  their  absence(s)  due  to 
CLC’s  strict  attendance  policy. 

The  first  two  “in  class”  statements:  “Now  I 
know new  words”  and  “Now  I under- 
stand  ” took  some  time  as  the  students  had  to 

decide  whether  “new  words”  meant  words  they 
had  never  heard  before  the  past  week  or  words/ 
concepts  which,  recently  reinforced,  were  now 
better  understood.  At  this  point,  students  referred  to 
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(and  some  copied)  their  classmates’  responses  to  be 
sure  that  they  were  on  the  right  track.  They  seemed 
uncertain  about  what  I wanted  from  them.  How 
much  writing  would  satisfy  the  teacher?  The  task 
was  becoming  increasingly  cumbersome. 

When  we  came  to  the  third  “in  class”  state- 
ment, “I  don’t  understand they  asked  if 

they  could  leave  this  space  blank.  Only  three 
students  wrote  in  this  space.  Answers  to  “This 
week  I liked  ” and  “This  week  I didn’t 

like ” seemed  to  me  to  be  biased  and  therefore 

uninformative.  When  asked,  “What  did  you  like?” 
most  students  answered,  “Everything!”  Half  of 
the  students  had  been  in  my  classes  for  over  six 
months,  and  we  had  shared  many  good  feelings. 
They  wanted  me  to  know  that  they  had  been  pleased 
with  the  classes.  No  one  admitted  that  they  didn’t 
like  something. 

For  the  questions,  “I  listened  to speak 

English”  and  “I  spoke  English  to ,”  I listed 

possible  answers  on  the  board.  Most  students  wrote 
some  variation  of  “myself,  the  teacher,  every- 
body, and  my  classmates.”  The  students  easily 
completed  the  last  “in  class”  statements:  “I 

read  in  English”  “I  wrote in  English” 

and  “I  needed  help  with .” 

The  students  also  had  no  trouble  filling  out  all 
four  of  the  “outside  class”  statements:  “This 

week  I spoke  English  to .”  “This  week  I 

listened  to speak  English”  “This  week  I 

read in  English”  and  “This  week  I 

wrote in  English.” 

Observations:  This  stage  of  the  field  testing 
was  much  more  time  consuming  than  the  goal- 
setting exercise.  I had  allotted  the  last  30  minutes  of 
class  for  a task  which  eventually  required  our  class 
to  go  12  minutes  over  time.  I suspect  my  vague 
introduction,  as  well  as  the  structure  of  the  form 
itself,  made  the  task  more  laborious  than  was 
necessary.  But,  filling  out  the  self-assessment  form 


was,  if  tedious,  a good  experience  for  the  students. 
The  “in  class”  section  required  the  learners  to 
recall  what  was  studied  during  class  and  their 
answers  reflected  what  information  I assume  was 
most  strongly  retained.  The  “outside  class”  sec- 
tion caused  them  to  consider  situations  in  which 
they  had  (or  had  not)  used  English  at  work,  home, 
and  in  their  community. 

Some  of  their  responses  were: 


Yelena:  At  Stop  & Shop.  Please  where  is 
the  milk?  Where  is  the  butter? 


a T»_r __.i t .... — * A — * — T 

mm.  ockjic  wiicu  1 went  t u uie  ut^iut , a 


needed  an  interpreter,  now  I do  it  myself! 


Jose:  I made  a phone  call  in  English  for  the 
first  time  yesterday.  I called  Connecticut  to 
order  something  for  work. 


Vitanie:  I read  English  signs  at  work. 


Students  mentioned  they  had  spoken  English  to 
family,  friends,  and  neighbors  at  home,  work,  and 
in  the  supermarket.  They  had  read  English  in  the 
dictionary,  at  home,  and  at  the  library.  One  student 
wrote,  “I  read  the  bus  name!”  They  had  listened 
to  English  on  the  radio,  on  television,  and  at  the 
supermarket.  Not  all  students  had  written  English 
outside  of  class,  but  those  who  had  had  done  so  at 
home  or  work. 

Once  completed,  the  forms  were  a great  source 
of  information  for  me.  The  students’  answers, 
particularly  the  answers  to  “I  need  help  with...,” 
offered  me  useful  knowledge.  As  a supplement  to 
the  assessment  of  each  learner  which  I gain  through 
“teacher  instinct,”  the  form  showed  my  students’ 
views  on  what  had  been  learned.  The  form  also 
helped  me  to  focus  on  my  teaching  techniques  and 
to  consider  ways  in  which  my  students’  learning 
could  be  more  enjoyable  and  efficient. 
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Teacher  C:  Weekly  Self-Assessment 

During  the  third  week  of  class,  I gave  the 
revised  assessment  form  to  my  students.  By  now, 
having  used  the  original  version  of  the  self-assess- 
ment, they  were  familiar  with  what  we  were  doing 
and  why  we  were  doing  it. 

Observations:  The  students  filled  out  the  new 
form  individually  and  with  far  greater  ease  then 
they  had  its  earlier  incarnation.  Compared  with  45 
minutes  spent  on  the  form  the  week  before,  the 
students  took  less  than  15  minutes  to  fill  out  the  new 
form.  While  the  students’  familiarity  with  the 
format  no  doubt  hastened  the  process,  the  improved 
form  clearly  helped  to  make  the  weekly  self- 
assessment  an  easier  and  more  enjoyable  task. 

Part  3.  The  Progress  Record* 

The  progress  checklist  is  to  be  filled  out  by  the 
class  as  a group  at  the  end  of  a unit  and  kept  as  a 
record  of  what  the  group  has  worked  on  and  how 


well  each  individual  feels  s/he  has  mastered  the 
topics.  The  teacher  and  class  determine  what  they 
have  worked  on  and  fill  in  the  left  side  of  the  list. 
The  students  then  check  whether  they  feel  they  have 
mastered  each  topic  or  whether  they  want  to  work 
more  on  this  area,  (figure  5,  appendix  8) 

Teacher  A:  Progress  Record 

Objective:  To  enable  the  student  to  look  back  over 
a session  and  record  what  s/he  learned,  with  an 
evaluation  of  whether  or  not  review  is  necessary. 

The  progress  report  activity  worked  well  with 
my  class.  Perhaps  my  students  were  getting  used  to 
the  idea  of  thinking  about  what  they  had  learned ; or, 
perhaps  the  checklist  format  was  easier  for  them 
than  completing  open-ended  statements.  To  begin 
theactivity,  we  established,  as  agroup,  whatwehad 
worked  on.  Each  student  contributed  something 
and  I recorded  the  words  on  the  board— I didn’t 
eliminate  any,  but  I did  classify  them  as  we  went 
along.  I wrote  all  the  body  parts  in  one  list,  all  the 
symptoms  in  a separate  list,  etc.  Then  we  talked 
about  the  categories  of  words  such  as  body  parts, 
symptoms,  and  medicines.  After  that,  students 
filled  in  all  the  blanks  on  the  progress  record.  Some 
wrote  general  categories;  others  wrote  the  vocabu- 
lary words  they  had  originally  contributed.  When 
everybody  had  filled  in  the  “We  worked  on...” 
column,  we  discussed  briefly  the  two  remaining 
columns,  “I  can  do  this”  and  “I  need  more.” 
Students  usually  checked  “I  can  do  this”  for 
individual  vocabulary  words  and  “need  more”  for 
categories.  A few  students  checked  “I  need  more” 
for  everything,  on  principle,  I suspect.  One  student 
did  not  complete  the  columns  at  all. 


* Teacher  C did  not  report  on  her  use  of  the  progress  record. 
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Teacher  B:  Progress  Record 

I reserved  25  minutes  at  the  end  of  our  final  class 
for  the  progress  record.  I wrote,  “What  we  worked 
on”  on  the  board  and  asked  the  class  to  name  the 
things  we  had  studied.  It  took  them  a few  minutes 
to  understand  what  I was  looking  for  and  then  they 
began  to  call  out  the  new  words  they  had  learned. 
I recorded  on  the  board  whatever  was  said,  trying 
to  get  everyone  to  offer  something  to  the  list.  I gave 
each  student  a progress  record  and  then  explained 
how  to  fill  it  out.  The  “I  can  do  this”  column 
seemed  confusing  because  of  what  we  had  studied. 
The  phrase  seemed  more  appropriate  for  talking 
about  a task  rather  than  a list  of  vocabulary  words. 
I changed  the  columns  to  read  “I  understand”  and 
“I  want  to  study  more.”  The  students  copied  from 
the  list  on  the  board  for  their  own  forms  and  then 
checked  the  appropriate  column. 

Observation:  Perhaps  what  is  important  about 
this  form  is  that  the  class  (collectively  and  the 
individually)  could  list  the  terms  that  we  studied. 

Status:  Ongoing 

The  process  of  defining  and  developing  assess- 
ment tools  at  the  Community  Learning  Center 
continues.  The  current  status  of  alternative  assess- 
ment at  CLC  is  "ongoing. " We  hope  this  account  of 
our  experiences  proves  useful  to  other  ESL  teach- 
ers. We  would  be  very  happy  to  hear  from  anyone 
who  has  used,  changed,  or  expanded  on  the  tools  we 
have  described  here. 

Karen  Ebbitt,  Priscilla  Lee,  Pam  Nelson,  and 
Joann  Wheeler  teach  at  the  Community  Learning 
Center  in  Cambridge. 
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Further  Adventures  in 
Alternative  Assessment: 
an  Annotated  Bibliography  (excerpt) 


Don  Robishaw 


n this  issue  of  Adventures  in  Assessment  we 

Xhave  taken  you  into  the  world  of  ongoing 
assessment— examining  the  process  and  the 
tools  and  techniques  practitioners  use  and  looking 
at  how  ongoing  assessment  can  benefit  and  serve 
the  interests  of  learners  as  well  as  teachers. 

Alternative  assessment  tools  are  designed  and 
used  to  show  what  students  know,  not  what  they  do 
not  know.  Alternative  assessment  is  alternative 
because  it  provides  choices  that  go  beyond  tradi- 
tional or  standardized  measurement.  Alternative 
assessment  empowers  learners  and  helps  them 
become  self-directed;  standardized  instruments  may 
disempower  learners  by  assuming  all  of  the  “au- 
thority” or  by  making  them  dependent  on  teachers. 

Alternative  assessment  is  ongoing  and  part  of 
the  curriculum,  not  imposed  or  added  on  at  the  end. 
An  alternative  assessment  component  may  contain 
a variety  of  tools,  including  writing  progress  check- 
lists, reading  progress  checklists,  learning  con- 
tracts, booklists,  miscue  analyses,  etc.  Depending 
on  their  developmental  level,  some  learners  can 
actually  create  their  own  self-assessment  tools. 

The  primary  purpose  of  alternative  assessment 
is  to  assist  students,  advance  learning,  and  improve 
instruction.  Ongoing  alternative  assessment  is 
continuous  and  not  separated  from  instruction. 

To  support  and  enrich  teachers’  endeavors,  the 
Research  and  Design  unit  of  SABES  has  compiled 
“Alternative  Assessment:  An  Annotated  Bibliog- 


raphy.” This  bibliography  is  intended  to  be  a 
resource  to  teachers  who  are  interested  in  learning 
more  about  the  topic  as  well  as  those  interested  in 
using  the  various  procedures  described  in  Adven- 
tures in  Assessment.  In  the  last  issue  of  Adventures, 
I reviewed  several  publications  that  related  to 
“getting  started”  with  alternative  assessment. 
This  time,  I have  excerpted  several  annotations 
relevant  to  ongoing  assessment. 

Sample  Annotations 

003  Arrastia,  M.,  Betancourt,  A.  & Wang,  D. 
(1990).  Report  on  Staff  Development  Project  on 
Alternative  Assessment:  Mothers  Reading  Program. 
New  York:  America  Reading  Council. 

This  selection  discusses  how  one  adult  literacy 
project  examined  standardized  testing  used  with  its 
students.  It  proceeds  from  the  premise  that  for 
evaluation  to  be  useful  it  must  help  learners  to 
evaluate  themselves,  so  that  they  can  be  in  full 
control  of  their  own  learning.  Standardized  tests 
are  often  used  as  gates  (exclusionary)  that  close  off 
certain  opportunities  to  learners.  Evaluation  and 
tests  often  contribute  to  learners  developing  a belief 
that  they  are  inadequate. 

Besides  the  short  narrative,  this  article  also 
includes  an  appendix  containing  samples  from 
student  portfolios.  Items  of  interest  in  the  portfolio 
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include  teacher-  and  student-made  tests  based  on 
student  generated  text. 

016  Nash,  A.,  McGrail,L.  & others.  (1989).  "Our 
Class:  A Weekly  Literacy  Ritual. " Talking  Shop:  A 
Curriculum  Sourcebook  for  Participatory  Adult 
ESL.  Boston:  University  of  Massachusetts. 

“Our  Class”  is  the  name  of  a weekly  class 
journal  which  documents  the  activities  of  a group  of 
adult  learners  in  Boston.  The  teacher  collects 
thoughts,  quotes,  factual  information  and  anything 
else  of  interest  and  writes  them  at  the  end  of  the 
week  for  the  class  to  use  as  a reading  and  writing 
exercise.  Eventually,  the  class  itself  compiles  the 
journal. 

The  authors  explain  their  method  and  discuss 
the  importance  of  ritual  in  the  literacy  classroom. 
Samples  of  “Our  Class”  summaries  are  included. 

026  Lawson,  L.  (1988).  Record  Keeping  in  Adult 
Education.  "Current  Issues  in  Teaching  ESL." 
Nicholas,  S & Hoadley-Maidment,  E.  Eds.  New 
York:  E.  Arnold. 

This  paper  discusses  the  theory  behind  keeping 
certain  kinds  of  records.  It  also  describes  some 
innovations  in  record  keeping  in  adult  education. 
Record  keeping  has  become  very  important  in  the 
field  of  adult  education  in  the  last  few  years, 
especially  since  the  onslaught  of  alternative  assess- 
ment strategies  and  wider  recognition  of  the  impor- 
tance of  independent  learning  theory. 

This  short  article  is  chock  full  of  alternative 
record  keeping  tools  and  instruments,  as  well  as  a 
brief  rationale  for  using  them.  For  the  "organiza- 
tionally challenged, " this  article  will  put  you  on  the 
road  to  getting  your  act  together. 


029  Soifer,  R.,  Irwin,  M.  & others.  (1990)  The 
Complete  Theory  to  Practice  Handbook  of  Adult 
Literacy:  Curriculum  Design  and  Teaching  Ap- 
proaches. New  York:  Teacher’ s College  Columbia 
University,  pp.  170-180. 

The  primary  purpose  of  this  chapter  is  to  give 
a brief  overview  of  seven  different  ways  to  assess 
adult  literacy  learners'  progress.  The  main  feature 
is  an  assessment  grid  that  analyzes  each  of  the  seven 
assessment  tools  that  follow: 

1.  Booklist 

2.  Spelling  and  vocabulary  graph 

3.  Anecdotal  records  (teachers) 

4.  Anecdotal  records  (students) 

5.  Writing  assessment 

6.  Reading  behavior  questionnaire 

7.  Attendance 

The  authors  contend  that  all  tools,  including 
those  mentioned,  should  be  used  in  ways  that  focus 
on  strengths,  progress;  and  applications  rather  than 
weaknesses,  deficiencies,  and  isolated  skills. 


Notes 

As  with  many  works  which  assemble  a collection  of 
readings,  this  bibliography  is  the  result  of  the 
efforts  of  several  people--from  the  development  of 
the  initial  idea,  to  the  search  and  collection  of  books 
and  articles.  I am  indebted  to  the  following  people 
for  their  assistance  in  this  project:  Loren  McGrail 
of  World  Education/S  ABES  who  supplied  me  with 
many  of  the  resources  and  Joan  Dixon,  the  Literacy 
Support  Initiative  Coordinator.  In  addition  the 
following  people  acted  as  reviewers:  Janet  Kelly, 
Director  of  the  Read/Write/Now  program;  Janet 
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Isserlis  of  the  International  Institute  of  Rhode 
Island;  and  the  following  University  of  Massachu- 
setts graduate  students:  Micheie  M.  Sedor,  Barbara 
Huff,  Susan  Schellenberger,  Haleh  Arbab,  Keyvan 
Kabastioun,  and  Ed  Graybill. 

The  complete  annotated  bibliography  contains  51  en- 
tries and  will  be  updated  on  an  ongoing  basis. 
Contributions  to  the  collection  are  welcome.  Spe- 
cifically we  are  looking  for  help  in  the  next  edition 
of  Adventures  in  Assessment:  Looking  Back.  Ar- 
ticles and/or  annotations  on  student/teacher  confer- 
ences, classroom  evaluation  and  program  evalua- 
tion are  welcome.  Please  contact  your  Regional 
SABES  Coordinator  or  Loren  McGrail  at  World 
Education  if  you  would  like  to  annotate  some  of  the 
collected  articles  and/or  add  new  ones. 


Adventures  in  Assessment,  Volume  2 


An  Invitation 

Letters/Responses 

The  next  issue  of  Adventures  in  Assessment  will 
feature  readers'  responses  to  the  articles  in  this 
volume. 

Perhaps  one  of  the  articles  reminded  you  of  a 
classroom  assessment  problem  you  faced  and/or 
solved.  Maybe  you  have  questions  or  additional 
information  about  the  tools  and  procedures  you  read 
about.  Or  perhaps  you  just  feel  like  conversing  with 
other  practitioners  who  are  looking  at  issues  of 
alternative  assessment. 

No  excuses  now!  We've  even  provided  you 
with  a response  mailer  (facing  page).- 

To  be  included  in  Volume  Three,  letters  must  be 
received  by 

July  17,  1992 

Articles 

Volume  Three  will  focus  on  "Looking  Back" 
activities  that  come  at  the  end  of  a cycle  and  that  help 
teachers  and  learners  reflect  on  what  they  have  and 
haven't  accomplished.  However,  we  are  accepting 
articles  that  address  any  of  the  three  components  of 
assessment:  initial,  ongoing,  and  end-of-cycle. 

If  you  would  like  to  submit  an  article,  call  Loren 
McGrail  at  (617)  482-9485.  Writer's  guidelines  are 
available. 

To  be  included  in  Volume  Three,  articles  must 
be  received  by 

June  19,  1992 
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Loren  McGrail 
Adventures  in  Assessment 
World  Education 
210  Lincoln  St. 

Boston,  MA  02111 

FAX:  (617)  482-0617 


Dear  Loren  and  Adventures  Staff, 


Adventurously, 
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Don't  forget 
the  postage! 


Loren  McGrail 
Adventures  in  Assessment 
World  Education 
210  Lincoln  St. 

Boston,  MA  02111 
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Appendix  1 - Suggestions  for  Individual  Reading  Conferences 


* A reading  conference  is  an  informal  meeting  between  a reader  and  teacher  to  talk  about 

the  independent  reading  a learner  is  doing.  The  discussion  can  include  the  experience  of 
reading,  the  story,  characters,  events,  and  any  problems  or  successes  the  reader  is 
encountering.  An  individual  reading  conference  can  also  be  used  to  do  an  informal  miscue 
analysis,  using  Alternative  Procedure  IV  and  the  Procedure  IV  Individual  Conference  Form 
which  is  reproduced  in  the  miscue  section  of  this  kit.  ’ 

* Use  a Reading  Conference  Record  form  to  make  notes  about  individual  reader’s 
conferences.  The  same  form  can  be  used  by  readers  to  keep  track  of  their  own  responses 
to  independent  reading.  The  Reading  Conference  Record  form  becomes  part  of  the  learner’s 
assessment  portfolio. 


* Ask  how  the  reader  chose  the  book  or  help  him/her  choose  something  they  are  interested 
in  reading. 


* Ask  how  the  reading  is  going.  Does  the  reader  find  it  easy,  a little  bit  difficult,  or  very 
difficult  to  read?  Why? 


* Is  the  reader  enjoying  the  reading?  Not  enjoying  it?  Why? 


* Is  there  any  character  in  the  story  the  reader  really  likes,  dislikes,  or  identifies  with? 
Why? 


* If  the  reader  is  not  finished  with  the  book,  what  does  he/she  predict  will  happen  in  the 
next  part  of  the  story? 


* What  does  the  reader  do  when  he/she  comes  to  unknown  words? 
(Suggest  alternative  strategies  if  needed) 


* Ask  the  reader  to  tell  you  something  about  what  they’ve  read. 


* If  the  reader  has  finished  the  story  or  book,  ask  if  he/she  would  recommend  it  to  others 
to  read.  Why  or  why  not? 


* Ask  what  they  think  they  would  like  to  read  next. 


0 
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Appendix  2 - Sample  Open  House  Assessment  Materials 


LEARNING  QUESTIONNAIRE 
Harborside  Open  House 

What  do  you  already  know?  Do  you  read  the  newspaper?  Then  you  can  read. 

Do  you  write  letters?  Then  you  can  write.  Please  answer  all  the  questions 
below  as  honestly  as  you  can.  Your  answers  can  help  both  you  and  the 
teachers  place  you  in  the  best  class  at  our  school. 

READING 

1 .  How  often  do  you  read? 

all  the  time  a lot  /^somet j,mes3  never 


2.  When  do  you  read? 

C^Sy  yourself 
-to  your  children 
-to  your  parents 
-at  your  job 
-in  school 


3.  What  do  you  read? 


(^newspaper  s^ 
t^stor  iesh 
-novels 
^magazines? 
-Bible 

-how-to  boo ks 
(^letters!) 
-comics 
-other : 


4.  What  do  you  read  at  work? 

-nothing 

-instructions 

-repair  manuals 

-applications 

-charts 

-blueprints 

<g£hej^ 


5.  How  comfortable  do  you 

a.  Reading  for  yourself 

b.  Reading  for  your  job 

c.  Reading  for  school 


feel . 

■ • • 

fine 

need 

help 

fine 

Cpkay^) 

need 

help 

fine 

need 

help 
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Appendix  2 (cont.) 


WRITING 

1.  How  often  do  you  write? 


never 


3.  What  do  you  write? 


-nothing 

fetter s/notes_to.  family ~ friends? 
(^shopping  lists j 

-diaries  ^ 

-business  letters 
-stories 

-poetry 

f^job  applications? 

-reports 

-dictation  (secretarial  skills) 
-other: 


MATH 

1 . How  often  do  you  use  math? 
always 


C~ a ^lotjsomet  imes 


never 


2.  When  do  you  use  math? 


-never 
-at  wor  k 


d£d 


JLancing  a checkbook 

-building 

<gX^tjrTng  a budget) 

-helping  children  with  homework 
-other; 
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READING  QUESTIONS 
Harborside  Open  House 


1.  Which  reading  did  you  choose? 

2.  In  a sentence  or  two,  tell  what  the  reading  was  about? 


3.  Could  you  read  five  pages  of  this  a night? 


4.  Do  you  think  this  is  a good  reading  level  for  you  to  start  on?  Do  you  think 
it’s  too  easy  or  too  hard? 

5.  Please  write  any  words  you  did  not  understand. 
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Appendix  2 (cont.) 


Jk 


*/  d*rn,  j£ 

^ 4sUJud£,  #*&y<><7®zZ/  suVte  s&r' 

AlO  y^uX^A^U UAO 

s^u£srnZ£[/ 
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MATH  ASSESSMENT 
Harborside  Opart  House 


9 


Directions: 

Please  circle  all  the  problems  on  this  page  that  you  know  you  can  do. 

Star  the  problems  that  you  have  done  before,  but  don’t  remember  how  to  do. 
Leave  blank  all  the  problems  you  can’t  do. 

DO  NOT  ANSWER  ANY  OF  THESE  PROBLEMS. 


/TTpat 


If  Pat  buys  a loaf  of  bread  for  49  cents  and  a jar 
of  peanut  butter  for  75  cents,  how  much  change 


( 1 ) $5.26 

(4)  $1.24 


(2)  $4.25 
(5)  $0.26 


(3)  $3-76 


o 

ERLC 


The  approximate  area  A oi/jhe  curved  surface  of  a 
right  circular  cylinder  is^rven  by  the  formula 
A = where  d is/he/Uia meter  and  h is  the  . 

height  of  the  cylinder,  yhnt  is  the  approximate 
area  of  the  curve**  surface  of  n^pgfTTcircular 
c y 1 i r^Tricaljn nif  3 feet  in  diameter  and  14  feet  high? 


( 1 ) 66 ft 

(2)  13Zsq.fr' 
( 3)  264  sq  ft 

( 4)  / 924  sq  ft 
( of  6,468  sq  ft 
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Appendix  2 (cont.) 


WHEN  YOU  HAVE  ANSWERED  ALL  THE  QUESTIONS,  TAKE  THIS  TO  A TEACHER 
VERY  MUCH . 

1 . Do  you  know  enough  now  to  choose  the  best  class  for  you? 

2.  If  you  do,  circle  the  class  which  is  right  for  you. 


THANK  YOU 

yrs 


EDP 


GED 


0 — 

ERJC72 
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Appendix  3 - Skills  Evaluation  Sheet 


NAME: 


DATE: 


skills/activities 


ccrrmen  ts/evaluat  ion 


O 
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Appendix  4 - Periodic  Pencil/Paper  Survey 


Name 


m o v . 6 /ia/fp. 

Date 


yesterday,  DJ  came  to  visit.  We  wrote  about  DJ.  I'J 

X f-'  ^*1 

r D.J.  works  with  Bill.  D.J.  is  a volunteer. 

( f^rU: 

D.J.  wants  to  know  about  our  class  so  he  can  write 
very  excellent  grants. 

D.J.  lives  on  the  east  side  of  Providence.  He  is  married, 
and  he  has  no  children.  He  got  married  in  1980. 

P/vV^  ‘ A/°  ^'r  . '* 


r~ 


We  also  talked  about  why  people  steal  things.  Last  week,  people 
stole  things  from  La's  house. 

Some  people  said  that  people  steal  because: 

— they  don't  have  mol^jy  and  they  can't  buy  things  in  the  store  so 
they  go  out  and  try  to  steal  from  other  people. 

— — because  they  need  money,  because  they  want  money  to  buy  drugs 

— because  they're  lazy  to  work. 

— because  they  are  poor  people. 

— because  they  are  bad. 


Why  do  so  many  people  steal  things? 
What  can  we  do  about  it? 

p-IMa  r rw/-\  r g o ; 

Qfr'r  EiHl  f/\ 


W/\ 

Vvf  (r\.r\/ 


. ^ S C \>r  0 o l - 1 1 •,  i*v , i £ 

^ m C Kt?  j)o  p -pvHrM  £mi  ds  ye  p fopc£  mm' r i 


* fsccm  I 


TV  m i /v  fk?  gWi  f 

*"  ~ ' — l j ^ I 

p°l  (M|  rK 

r h £ a/) 


f rf\e  /A  our-  i-rovs Cc3  o 

aImw 


; ^ i K p >tv_  ^ 

jte  urv>ebt  /i^  ie<l  ht  ii~t, 

FvZ  TRFvTT 


J ^yrv'^vV  t . 

Are  laws  about  stealing  different  in  America  than  they  are  in  your 

country.  LOtfyi-  7W_  ^ j 

Wr^_  hi  ^ I AA(^K 
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Appendix  5 - Goal-setting  exercise  "survival  areas" 


DOCTOR, 

□ 


- 


Food 

SHOPPING 

n 


MOW  ARE  You? 

■Feeling 

a 


Toes 


WOP.K 

a 


to  ot/r  au?N£, 
bOTTtp  utJES  AND 

i>i^rfc\£>utep  n> 

u:d6K*T£  A£  a $>/y ct. 

^ ^ sepARArg-  cAgPs. 
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SHOPPING 
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Appendix  6 - Weekly  Self-Assessment,  Form  A 


Fag.A'l  A 

Name  ‘ ~ Week  ending  ‘ 

How  many  hours  in  class  this  week?  ■ . 

***********************  *********************************************************** 

IN  CLASS 

1.  Now  I know  these  new  words: . ± . __ 


Now  I understand 

- 

I don’ t- understand 

In  class  this  week  I liked 

• 

• . 

# 

5.  In  class  this  week  I didn’t  like 


6.  In  class  I listened  to  

7.  In  class  I spoke  English  to 

8.  In  class  I read  

9.  In  class  I wrote 

10.  In  class  I need  help  with  _ 

**************************************** 

OUTSIDE  CLASS  ’ ‘ ‘ ' 

This  week  I spoke  English  to  ■ _ ' 

This  week  I listened  to  speak  English. 

This  week  I r6ad  ; in  English. 

This  week  I wrote  ■ ' .in  English. 

Adventures  in  Assessment , Volume  2 
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1. 

2. 

3. 

A. 


speak  English. 


in  English. 

_in  English. 


Appendix  7 - Weekly  Self-Assessment,  Form  B 


M Am 


=Nv;,  c-p  classes 

lasf  uv^K  ■ 


Ni  CLASS 


1.  Now  I understand 


2.  In  class,  I need  help  with 


Outs  idle  of 

This  week  I spoke  English 


4. 


I listened  to 


5. 


I read  English  ■') 


speak  English* 


6.  I wrote  English 
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Appendix  8 - Progress  Record 


weeks 


We  worked  on 

1. _ 

2.  

3. ^ 

4.  

5.  

6. 

7.  

8. 


PROGRESS  RECORD* 

■ ' I 

I can  do  this 


O 

ERIC 
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this  volume  of  " Adventures  in  Assessment.  ” 

The  first  three  volumes  of 44 Adventures  in  Assessment*’  present  a comprehensive  view  of  the  state  of 
practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners.  Volume  1,  Getting 
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Foreword 


Around  this  time  last  year,  just  when 
the  last  leaves  had  fallen  off  the  trees, 
Volume  1 of  Adventures  in  Assess- 
ment was  Dublished.  Hot  off  the  photocopy 

* • - - 

machine,  it  landed  in  the  hands  of  a group  of 
practitioners  from  across  the  state  who  were 
participating  in  the  first  Massachusetts  adult 
basic  education  annual  conference,  Network  ’91. 

That  first  issue  of  Adventures  looked  at 
learner-centered  approaches  to  assessment 
featuring  tools  and  strategies  for  “getting 
started."  Practitioners  wrote  about  start-up 
activities,  intake  processes,  and  tools  used  to  get 
a sense  of  students’  interests  and  goals.  In  the 
spring,  Volume  2 appeared,  featuring  writing 
about  “ongoing”  assessment  procedures  and  the 
ways  we  document  learning  as  it  takes  place, 
including  using  tools  such  as  journals,  portfo- 
lios, logs,  and  checklists. 

Now,  it  is  late  fall  again-time  for  Network 
’92  and  time,  also,  for  Volume  3 of  Adventures 
in  Assessment.  In  this  issue  we  turn  to  the  tools 


and  procedures  used  at  the  end  of  a cycle  or 
semester  when  learners  and  teachers  are  “look- 
ing back”  at  their  achievements  and  progress.  Of 
course,  the  process  doesn’t  end  with  looking 
back:  the  leaves  fall,  the  ground  freezes  then 
thaws,  the  daffodils  bloom  and  wither,  the  days 
lengthen  then  shorten  again.  Learning  never 
stops,  so  this  issue  of  Adventures  also  includes 
articles  on  “starting  again.” 

This  year,  then,  Adventures  in  Assessment 
contributors  have  taken  you  on  a journey  from 
beginning  to  beginning.  Some  of  the  people  in 
that  first  group  of  readers  will  come  back  to 
Network  ’92  as  writers  as  well  as  readers  of  this 
one  year  old  publication.  We  are  proud  of  their 
accomplishments  and  look  forward  to  hearing 
from  other  practitioners  who  may  never  have 
written  about  and  shared  their  ideas  before. 

We’re  pleased  to  introduce  with  this  issue, 
three  new  continuing  features  which  we  hope 
will  further  extend  the  dialogue  on  assessment 
in  adult  education.  “What  Counts?”  will  keep 


readers  up-to-date  on  math  assessment,  “Voices 
from  the  Field”  will  feature  interviews  with 
practitioners  and  learners,  and  “Letters”  offers 
every  reader  the  opportunity  to  respond  infor- 
mally to  the  opinions  and  practices  they  read 
about  in  this  journal. 

Write  or  call  us  with  your  thoughts,  com- 
ments, and  ideas.  We  would  especially  like  to 
hear  from  those  of  you  who  have  adapted  some 
of  the  tools  presented  in  Adventures  to  suit  your 
own  classroom  or  program’s  needs. 

-•  Laura  Purdom,  Editor 


Please  note  the  corrections  from  Volume  2 
listed  before  the  appendices  in  this  volume. 
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Introduction: 
Looking  Back,  Starting  Again 


Looking  Back 

Because  of  the  ongoing  nature  of  this  pro- 
cess-learners learning  at  various  rates,  and 
with  varying  degrees  of  speed  and  ahilities- 
our  attempts  to  reflect  upon  progress  at  the 
end  of  the  cycle  are  really  part  of  a larger 
circular  process  of  learning  we  never  really 
finish,  (Janet  Isserlis,  p.10) 

Looking-back  assessments  are  done  at  the 
end  of  a cycle.  They  can,  as  Janet  Isserlis 
states  above,  be  a process  of  learning 
about  learning  for  teacher  and  student  alike.  In 
general,  they  serve  five  main  purposes: 

• to  indicate  the  levels  of  proficiency  learn- 
ers have  attained  in  language  and  literacy; 

• to  indicate  if  learner  goals,  needs,  and 
interests  have  been  met; 

• to  document  changes  that  have  occurred  in 
learners’  lives  both  inside  and  outside  the 
classroom; 

• to  provide  evidence  of  a program’s  success; 
and 

• to  provide  assessment  data  that  has  been 
aggregated  across  components,  summa- 
rized, and  made  comprehensible  to  an 
audience  external  to  a program  (perfor- 
mance review). 

Depending  upon  its  philosophy,  focus,  and 


approach  to  language  and  literacy,  a program 
may  focus  on  any  or  all  of  the  above  objectives. 
Programs  that  aspire  to  a more  participatory 
form  of  assessment  will  involve  learners  during 
both  the  development  of  the  assessment  strategy 
and  during  its  implementation.  The  following  is 
a list  of  strategies  or  procedures  programs  in  this 
volume  have  used  to  meet  some  of  these  objec- 
tives: 

Student  self-evaluation  in  which  learn- 
ers use  surveys,  checklists  or  narratives 
to  show  changes  that  have  occurred  in 
their  goals,  interests,  and  needs  for 
literacy  (Isserlis,  Reddy); 

Reviewing  documentation  of  classroom 
observations  (teacher  logs)  to  summarize 
progress  (Isserlis); 

Class  Evaluations  in  which  groups  of 
learners  provide  feedback  on  the  class, 
discussing  what  they  liked  and  didn’t  like 
(Cason,  Reddy);  and 

Program  evaluations  in  which  students 
from  various  classes  come  together  to 
discuss  programmatic  issues  like  class 
time,  grouping,  and  child  care  (Reddy, 
Cason). 

There  are  a two  other  procedures  not  refered 
to  by  the  authors  in  this  journal  which  are 
worth  mentioning: 


Peer  interviews  in  which  students  ask  each 
other  questions  they  have  generated  individually 
or  collectively,  and 

Student/teacher  conferences  in  which 
questions  which  have  been  asked  at  intake  are 
reviewed  for  changes  in  literacy  practice,  literacy 
strategies,  meta-cognitive  awareness,  increased 
range  of  literacy  materials,  support  systems  used 
or  the  support  provided  to  others. 

There  are  many  reasons  why  these  strategies 
and  looking  back  assessment  strategies  in  general 
have  not  been  developed  or  implemented  as 
fully  as  initial  assessments  or  ongoing  assess- 
ments. The  biggest  reason  is  time,  a precious 
commodity  for  the  majority  of  programs  operat- 
ing with  mostly  part-time  staff  on  a shoe  string 
budget  which  barely  covers  prep  time  not  to 
mention  time  to  evaluate  learners  holistically. 
However,  I believe  there  is  another  reason,  one 
which  is  even  more  basic.  All  of  us-  administra- 
tors, counselors,  teachers,  and  students-find  it 
difficult  to  talk  about  literacy  and  language 
development  beyond  talking  about  skills.  We 
are  all  new  to  thinking  about  learning  as  a 
process  and  not  just  a product.  What  Isserlis 
claims  is  true  for  many  leamers-that  they  have 
“an  innate  sense  of  their  own  language  move- 
ment but  the  expression/verbalization  of  that 
progress  may  not  be  within  the  frameworks 
they  have  developed  or  use  in  describing  learn- 
ing”-is  probably  true  for  practitioners  as  well. 
We  are  just  at  the  beginning  of  developing  our 
own  framework  and  don’t  always  know  how  to 
translate  our  thoughts,  hunches,  and  ideas  about 
what  constitutes  progress  into  practice,  tools, 
and  procedures  that  measure  and  document 
what  we  believe  to  be  real  indicators  of  growth 
and  change. 


Starting  Again 

As  Ann  Cason  writes  in  this  volume  (p.  14), 
the  all-program  evaluation  is  “a  tool  that,  while 
helping  us  to  look  back,  is  really  just  a begin- 
ning.” What  we  count  as  progress  or  success  at 
the  end  of  a cycle  should  have  been  noted  or 
collected  as  data  at  the  beginning  of  a cycle. 

Which  aspects  of  language  and  literacy 
should  be  addressed  by  a program  depends  on 
which  elements  the  program  wants  to  empha- 
size. There  are  a number  of  indicators  for  lit- 
eracy success  that  programs  might  focus  on.  In 
general,  they  can  be  categorized  as  evidence  of 
progress  related  to  language  and  literacy  devel- 
opment and  evidence  of  program  success  in  non- 
linguistic  domains.  (Wrigley,  p.  139) 

When  developing  start-up  or  intake  activi- 
ties, some  of  the  linguistic  and  non-linguistic 
indicators  of  success  to  look  for  include: 

• getting  a sense  of  what  learners’  needs  and 
goals  are; 

• finding  out  how  learners  use  literacy  in 
their  lives  inside  and  outside  the  classroom 
(in  their  community  or  at  the  workplace); 

• understanding  what  literacy  means  to 
learners  or  how  learners  judge  their  own 
capabilities; 

• finding  out  how  learners  go  about  making 
sense  of  print,  what  strategies  they  do  or 
do  not  use;  and 

• documenting  what  learners  can  do  with 
literacy. 

The  activities  and  tools  used  to  document 
the  kinds  of  progress  listed  above  include: 

Informal  Interviews 

(Schmitt,  et  al;  Jackson,  et  at) 


O 
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Goal-setting  Activities 

(Gluckman,  et  al\  Schmitt,  et  at) 

Reading  Samples 

(Jackson,  et  at) 

Writing  Samples 

(Jackson,  et  at,  Gluckman,  etat) 

Language/Literacy  Inventories 

(Gluckman,  et  at) 

Focus  Groups 

(Jackson,  et  at) 

Of  all  these  strategies,  the  language/literacy 
inventory  is  the  most  difficult  to  develop  and 
implement,  yet  it  may  be  the  procedure  that 
gives  both  learner  and  teacher  the  most  concrete 
evidence  or  “base-line  data”  on  how  a learner 
uses  language  and  literacy  in  his  or  her  everyday 
life.  It  is  the  one  tool  that  may  help  us  under- 
stand how  what  we  do  in  the  classroom  mani- 
fests itself  outside  the  classroom  in  the  family, 
the  community,  or  the  workplace. 

As  teachers  begin  to  think  about  their 
assessment  experiments,  they  sometimes  go 
through  a process  of  self-reflection.  This  kind 
of  reflection  may  be  disquieting,  but  I believe  it 
is  essential  to  teacher  research.  Amy  Gluckman 
describes  it  best  when  she  says  that  her  work  in 
assessment  “...has  probably  forced  me  to  ac- 
knowledge that  I don’t  believe  as  completely  in 
a student-centered  approach  as  I might  have 
thought.  There  are  some  things  that  I want  my 
students  to  learn  and  some  goals  that  I want 
them  to  have  whatever  they  think  about  it.”  (p. 
*)  Gluckman  is  reminding  us  that  there  is  not 
always  a clear  relationship  between  the  needs  of 
the  students  and  the  goals  of  the  program  and 
that,  in  fact,  they  can  even  be  at  odds  with  each 
other.  However,  a participatory  approach  to 


assessment,  like  a participatory  curriculum 
process,  must  start  with  valuing  what  learners 
want  and  need  literacy  for.  Starting  with  learn- 
ers’ goals,  needs,  and  interests  does  not  mean 
programs  have  to  abdicate  their  own  goals.  It  is 
a process  of  negotiation.  What  has  been  missing 
in  most  traditional  forms  of  learner  assessment 
is  even  asking  the  question. 

Re-assessing  assessment,  putting  into  practice 
a more  participatory  approach,  challenges 
learners  and  teachers  at  their  most  basic  level. 
That's  the  way  it  should  be. 

- Loren  McGrail 
Editor 
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What  Happened  to  Rosalie?: 
Thoughts  at  the  End  of  a Cycle 


by  Janet  Isserlis 

This  article  describes  a process  of  reflec- 
tion and  analysis  at  the  end  of  a cycle 
during  which  I facilitated  two  classes  at 
the  Literacy/ESL  Program  at  the  International 
Institute  of  Rhode  Island  (IIRI).  It  also  reflects 
my  concerns  about  learners’  progress.  These 
concerns  are  felt  throughout  the  course  of  any 
cycle  and  pertain  to  issues  regarding  learners  and 
the  connections  they  make  between  literacy  and 
language  laming  in  the  classroom  and  beyond 
that  context.  Initially,  I want  to  know  what 
learners  already  know  and  to  help  strengthen 
and  increase  their  abilities  with  literacy.  As  the 
cycle  continues  this  impetus  fuels  continued 
interaction  around  literacy  and  language,  and 
finally,  when  the  cycle  is  finished,  I need  to 
learn  what  has  happened  and  what  hasn’t  hap- 
pened in  order  to  continue  refining  the 
unending  process  of  learning  from  and  with 
learners. 

The  Setting 

The  ESL/Literacy  Program  at  the  Interna- 
tional Institute,  in  the  words  of  its  educational 
director,  “creates  a place”  for  those  learners  who 
can’t  really  thrive  in  regular  ESL  classes.  Not 
only  is  print  and  the  ability  to  interact  with 


print  a barrier  to  their  successful  participation, 
but  the  constraints  of  many  mainstream  ESL 
programs  present  barriers  as  well.  Regular, 
punctual  attendance  is  not  always  possible  for 
our  learners.  In  most  ESL  programs,  where 
waiting  lists  are  long,  attendance  policies  would 
quickly  eliminate  most  of  our  students.  Tuitions 
would  not  be  payable — even  the  nominal  fees 
charged  by  the  Institute  in  order  to  keep  its 


"... I need  to  learn  what  has 
happened.. .in  order  to  continue 
refining  the  unending  process  of 
learning  from  and  with  learners.  " 


other  programs  afloat.  While  learners  in  other 
programs  face  similar  barriers,  for  our  learners, 
the  extra  latitude  in  attendance  policies  and  the 
extra  support  around  and  beyond  literacy 
provide  important  first  steps  in  helping  them 
assume  greater  responsibility  for  their  own 
laming— initially,  in  accessing  the  class,  later  in 
developing  ability  with  language  and  literacy. 


The  author  is  indebted  to  Rosalie  Francois,  Sara  Smith,  and  Fran  Collignon. 
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The  Learners 

Rosalie  Franjois  finished  the  school  year 
on  June  19.  She  had  been  studying  in  the  Lit- 
eracy/ESL  program  at  IIRI  since  the  previous 
November.  Two  days  prior  to  our  last  class  I 


"She  said  she  had  felt  ashamed 
when  she  first  came  to  school 
because  all  she  had  been  able  to 
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she  is  a ' miracle . 


had  read  aloud  to  her  a paragraph  describing  her 
initial  entry  into  literacy  learning  (Adventures  in 
Assessment , Volume  II,  p.  41).  Reading  the 
passage,  I felt  apprehensive.  Rosalie  had  given 
me  permission  to  write  about  her  earlier  in  the 
year,  but  how  would  she  feel  about  actually 
hearing  her  strengths  and  weaknesses  being 
analyzed? 

Rosalie  was  delighted.  I gave  her  a copy  of 
the  article,  asking  if  she  would  like  to  take  it 
home  to  her  husband.  This  she  did.  She  re- 
ported during  our  last  session  that  her  husband 
was  also  very  happy  about  the  article,  as  well  as 
about  the  progress  she  made  during  the  months 
she  attended  classes.  We  have  spoken  at  length 
about  her  feelings  when  she  started  the  program 
and  about  how  those  feelings  have  changed 
during  the  course  of  the  several  months  she  was 
in  school.  She  said  she  had  felt  ashamed  when 
she  first  came  to  school  because  all  she  had  been 
able  to  write  was  her  name.  She  says  that  she  is  a 
“miracle.*  She  has  developed  a core  sight  word 
vocabulary  as  well  as  a few  initial  strategies  for 
dealing  with  words  she  cannot  decode.  During 


the  last  week  of  class  we  had  worked  on  address- 
ing letters.  Everyone  learned  where  and  how  to 
write  her  return  address,  where  the  stamp  goes 
and  where  the  address  is  written  as  well.  I in- 
vited learners  to  write  me  a letter  sometime 
during  the  summer  if  they  wanted  to.  Not  even 
a week  later,  Rosalie  had  written  me  a lovely 
letter. 

Rosalie’s  classmates  were  native  speakers  of 
Spanish,  French,  Creole,  and  Hmong.  For  the 
most  part,  the  class  was  comprised  of  women. 
(Three  men  studied  during  the  course  of  the 
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all  women).  Rosalie  and  Bacilia  were  the  most 
consistent  in  their  attendance  and  the  ones  with 
the  most  basic  level  literacy  skills.  This  class  met 
five  days  weekly,  for  two  and  a half  hours  in  the 
morning.  The  bulk  of  our  work  together  was 
generated  through  language  experience  stories 
which  were  typed  up  to  be  read  the  following 
day.  From  these  stories  we  built  strategies  for 
using  phonetic  cues  and  reading  new  sentences, 
as  well  as  continuing  ongoing  work  with  dates, 
days,  and  high  frequency  words.  We  also 
worked  from  photos  and  objects  that  the 
women  brought  in  to  share  with  each  other.  An 
ongoing  project  centered  around  reading  about 
and  discussing  homelessness,  culminating  in  a 
visit  to  a shelter  for  which  we  had  collected 
money  throughout  the  winter  and  spring. 

A few  of  the  women  who  were  more  ad- 
vanced than  the  others  chose  to  remain  in  the 
class  as  peer  helpers.  Eventually,  everyone  was 
able  to  help  everyone  else  at  various  times— 
from  helping  to  transcribe  a story  on  the  board, 
to  knowing  and  calling  out  a word  that  someone 
else  was  unable  to  decode.  As  a group,  the 
beginners  made  very  slow  but  steady  progress 
throughout  the  year.  Those  women  who  were 
already  able  to  read  and  write  strengthened  those 
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skills  they  did  have  by  participating  in  the  class 
and  by  working  with  other  learners  whose  skills 
were  less  developed. 

In  addition  to  Rosalie’s  class,  I worked  with 
another  group  of  learners  for  two  hours  daily, 
four  days  a week  in  the  early  evening.  This 
group  was  on  the  whole  more  advanced  than 
Rosalie  and  her  beginning  level  classmates,  all  of 
whom  were  new  to  the  program  this  past  year. 

In  contrast,  most  of  the  evening  students  had 
been  in  class  with  me  or  with  Fran  Collignon 
for  at  least  six  months.  Those  few  who  were  not 
returning  from  the  previous  year  were  either 
friends  or  family  of  current  students  or  had  been 
on  a waiting  list  generated  at  the  start  of  the 
school  year. 

Although  we  used  a great  deal  of  language 
experience  writing  in  this  group,  we  also  did  a 
fair  amount  of  reading  for  new  meaning— using 
texts  from  Voices,  various  textbooks  (readers  for 
the  most  part)  as  well  as  homework  based  on 
learner-generated  writing  from  class.  The  turn- 
over of  students  from  this  class  was  greater  for  a 
number  of  reasons:  we  added  a third  teacher  to 
the  staff  in  the  spring,  hoping  to  help  my  ad- 
vanced learners  make  the  transition  from  the 
ESL/Literacy  program  to  the  English  Language 
Center  (the  agency’s  mainstream  ESL  program) 
in  the  fall.  Additionally,  learners  came  in  from 
Fran’s  more  basic  level  class  and  others  left 
because  of  health  or  family  constraints.  One 
other  difference  between  this  class  and  the 
morning  session  was  that  the  learners  had  more 
oral/aural  English  at  their  command.  Although  I 
spoke  Spanish  and  French  when  necessary,  it 
was  far  more  difficult  to  get  the  morning  stu- 
dents to  speak  English  than  it  was  with  the 
evening  students,  probably  because  there  was  a 
sizeable  group  of  Hmong  learners,  and  only  a 
minority  of  Haitians  and  Spanish  speakers  in  the 


evening  class.  The  use  of  learners’  languages  in 
the  classroom  (Does  it  help?  Is  it  a disservice?  Is 
there  a balance  to  be  struck?)  is  one  of  the 
factors  I consider  when  stepping  back  from  the 
past  cycle  in  order  to  try  to  gain  a sense  of  how 
and  why  things  worked  and  did  not. 

The  Process 

When  examining  learners  and  assessment, 
Rosalie  is  easy.  For  her,  talking  about  her 
progress  comes  as  easily  as  breathing.  Her  aural/ 
oral  abilities  are  such  that  this  kind  of 
metawork1  is  dynamic  and  endlessly  illuminat- 
ing. She  is,  however,  atypical  of  most  adult  ESL/ 
literacy  learners.  Not  only  is  her  ability  to 
express  herself  in  English  unusual,  she  also 
accepts  that  talk  about  literacy  is  important.  She 
is  justifiably  proud  of  her  accomplishments  and 
positively  beams  when  discussing  them.  Her 
explicit  acknowledgement  of  her  progress  is  not 
typical,  particularly  of  those  learners  from 
southeast  Asia,  with  whom  I have  worked.  The 
notion  of  stepping  back  to  look  at  progress  over 


"When  examining  learners  and 
assessment , Rosalie  is  easy.  For 
her , talking  about  her  progress 
comes  as  easily  as  breathing. " 


a period  of  time  is  a very  school-like  idea.  Those 
of  vis  who  went  to  school  in  this  country  knew 
that  report  cards  would  come  quarterly.  Teach- 
ers would  chastise  those  of  us  “not  working  to 
our  fullest  potential”  and/or  would  praise 
improved  handwriting  and  comportment.  Our 
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nam 

1.  w**n  did  you  0*910  to  itudy  u>  •* 
j.  *oi*n  did  you  firat  co«*  to  tftit  «ct»ool> 
j.  la  Aoytiuog  diffaroet  for  you 


«.  tfhot  do  you  lia*  *oout  acftoolJ 


S,  What  doa't  yoo  iu*?  *0  yew  wbz  to  ena»*w> 


plaaaa  eitcio 

t.  ia  reading  aaaiar  for  you  now? 

7,  la  writing  aaaiar  fox  you  now? 

I.  la  apaakiog  aaaiar  for  you  now? 

>.  la  it  aaaiar  to  udnarataad  Engliao  now? 


yao  ao 
yaa  no 
yaa  no 
yaa  ao 


Vlaaaa  wtita  anytoiag  ala*  you  want  to  aay  aOout  ■cftool. 


a littla  a lot 
a littla  a lot 
a littla  a lot 
a littla  a lot 
or  adout  1 taming  1 


FigureI 


own  self-assessment  abilities  often  developed 
after  high  school— when  we  began  to  choose 
what  (or  if)  to  study.  We  may  have  known  early 
on  that  we  were  bad  at  math  or  good  in  English, 
but  for  many  of  us,  our  own  entr6  into  assess- 
ment was  through  those  assessments  levied  by 
our  teachers.  As  educators  now,  we  may  reflect 
back  and  find  that  we  have  a range  of  reactions 
to  our  previous  schooling.  Our  own  sense  of 
how  we  know  about  learning  and  progress  has 
evolved  from  a position  of  relative  strength— 
we’ve  been  literate  learners  for  years. 

For  learners  with  little  prior  schooling  or 


from  cultures  where  teachers  dictate  what  occurs 
in  a classroom,  the  concept  of  self-assessment 
may  be  difficult  to  grasp.  Surely,  many  learners 
have  an  innate  sense  of  their  own  movement 
with  language  and  literacy,  but  the  expression/ 
verbalization  of  that  progress  may  not  be  within 
the  frameworks  they  have  developed  or  use  in 
describing  learning.  Not  only  is  abstract  (gen- 
eral) self-assessment  sometimes  difficult,  even 
acknowledging  one’s  ability  to  accomplish  a 
specific  task  is  often  hidden  by  layers  of  cultural 
patterning.  Many  Hmong  women,  for  example, 
truly  readers,  will  say  “I  don’t  know"  when 
asked  to  read  aioud  in  class.  My  colleague  Fran 
Collignon,  who  has  worked  extensively  with 
Hmong  learners  in  community  and  educational 
settings,  tells  me  that  this  is  largely  bound  to 
women’s  roles  within  the  Hmong  culture. 

Many  of  the  other  learners  with  whom  I’ve 
worked,  when  asked  “What  do  you  think  you 
learned  this  year?”  tell  me  that  I’m  a good 
teacher,  I believe,  to  show  me  respect  and  ex- 
press appreciation.  They  do  not  tell  me  what 
they  have  learned,  however.  Others,  being 
modest,  tell  me  “nothing"  has  changed  for  them 
during  the  course  of  the  year.  The  pencil/paper 
work  we  do,  (see  Fig.  1)  as  well  as  ongoing  brief 
and  later  extended  discussions,  tell  me  that  some 
learners  do  not  necessarily  give  the  same  valence 
to  this  talk  about  learning  and/or  have  yet  to  be 
able  to  actually  feel  enough  of  their  own 
progress  to  be  able  to  step  away  from  it  and  say, 
“Yes,  before  this  year,  I couldn’t  read  the  sched- 
ule at  work.  Now  I can”  or  “I  still  don’t  read 
anything  at  home,  but  I feel  better  in  school.  I 
believe  that  this  metaprocess  is  viewed  in  differ- 
ent ways  by  different  learners,  and  that  factors 
such  as  prior  exposure  to  schools  or  progression 
from  one  level  to  another  in  their  current 
schools  might  have  something  to  do  with  this. 
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Discussions  about  progress  may  inevitably  be 
connected  in  learners’  minds  to  discussions 
about  a teacher’s  merits.  Not  only  do  cultural 
patterns  of  modesty  come  into  play,  other 
assumptions  about  what  teachers  need,  want  to, 
or  should  hear  are  also  involved.2 

Throughout  the  course  of  the  cycle  (from 
September  to  June)  with  both  groups,  I tried  to 
make  learners  more  consciously  aware  of  their 
learning  processes.  When  someone  was  able  to 
write  an  entry  in  a dialogue  journal  without 
assistance,  I would  point  it  out.  I would  encour- 
age someone  else  having  difficulty  reading  a line 
of  print  to  go  ahead  to  the  next  word  and  then 
try  to  figure  out  from  the  meaning  what  that 
word  might  be.  I would  ask  people  who  were 
clearly  making  progress  if  they  thought  they 
were  progressing.  Sometimes  my  questions  were 
met  with  polite  nods;  at  other  times  cultural 
patterning  took  over;  occasionally  someone 
would  agree  that  they  were  able  to  do  some- 
thing they  had  previously  been  unable  to  accom- 
plish. 

With  the  pencil/paper  surveys,  I try  to  elicit 
specific  information  from  learners.  I am  learning 
that  such  surveys,  at  least  at  present,  in  and  of 
themselves  are  occasionally  useful,  as  part  of  the 
necessarily  ongoing  assessment  process.  The 
surveys  themselves,  however,  seem  to  be  artifi- 
cial. These  surveys  ask  learners  to  think  and 
write  about  progress  after  six  or  nine  months— 
in  terms  somewhat  removed  from  the  everyday 
language  we  used  to  talk  about  learners’  literacy. 
However,  it  was  the  learners  themselves  who 
helped  me  to  see  in  a very  concrete  manner  the 
ways  in  which  independence  with  literacy  can  in 
fact  improve  one’s  outlook  on  day  to  day  life. 
These  insights  did  not  emerge  through  surveys. 

Bacilia’s  progress  is  evident  to  Rosalie,  who 
encourages  her  often.  It’s  apparent  to  me,  and  I 


Evening  Students:  The  Survey 

• Is  anything  different  for  you  now? 

“No  for  me  no  is  different  nothing.  “ 

♦ 

° different  country  for  me  different  people  for 
me  and  different  food  for  me  different 
school “ 

❖ 

“before  I understand  nothing,  now  I under- 
stand small" 

❖ 

“now  I learn  more  than  before.  Some  things 
1 can  do  alone.  Some  stories  from  my  friend  I 
can  read;  road  signs  a little  bit.  * 

❖ 

“Me  siento  diferente  en  mi  escuela  porque  he 
aprendido  cosas  que  no  sabia  y me  gusta  a mi 
profesora  tanto  como  la  que  teniqs  ante " 

♦ 

“different  country  in  people  in  school * 


point  out  to  her  the  difference  in  the  way  her 
print  looks  since  she  began  in  January  and  the 
way  it  looks  now,  months  later.  Bosilia  herself 
says  she  feels  happy  because  she  knows  some 
English  and  feels  a bit  more  capable  of  dealing 
with  office  appointments  on  her  own.  Her 
attendance  was  virtually  perfect;  only  clinic 
appointments  and  the  odd  day  or  two  found  her 
missing  from  class. 

For  Surprise,  the  gradual  progress  I’d 
observed  was  translated  (literally,  by  Rosalie, 
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from  Creole  into  English):  “She  didn’t  know 
what  to  do  [before];  not  her  name.  She  can  spell 
her  name,”  Surprise  can  also  follow  along  on  a 
page  when  someone  is  reading,  and  can  work 
her  way  across  a line  of  print.  For  now,  she 
relies  on  memory  and  only  slightly  on  actual 
print  for  cues  to  decoding.  Between  her  state- 
ment through  Rosalie,  her  affect  (she  attends 
more  frequently;  she  is  affectionate  and  warm 
where  she  was  once  silent  and  withdrawn),  and 
the  actual  strides  with  following  print,  I see  that 
some  foundation  has  been  laid.  Since  I started 


“ These  are  small  steps,  surely,  but 
important  ones  in  the  context  of 
the  ongoing  events  in  the 
classroom  and  the  small  tenuous 
bridge  we  try  to  build  between  it 
and  the  world  beyond.  “ 


writing  this  essay,  I’ve  received  a letter,  written 
by  Surprise’s  daughter  (“Because  of  my  daughter 
she  can't  print  and  I can  copy  maybe  next  time  I 
will  try  my  best  to  write  you  something”).  In 
her  letter,  Surprise  has  her  daughter  write  “I 
know  how  to  write  my  name,  take  a bus 
[Rosalie  showed  her  how  to  take  the  bus]  A,B,C 
so  on  and  so  forth...I  know  I can  read  even 
though  it’s  not  much  it’s  still  something  for 
me.”  (See  Fig.  2) 


Everyone  in  the  morning  class  can  now 
write  the  date  at  the  top  of  the  blackboard 
before  class  begins.  They  can  all  go  to  the  board 
to  transcribe  something— with  varying  degrees 
of  assistance  from  others.  They  are  all  willing  to 
try.  These  are  small  steps,  surely,  but  important 
ones  in  the  context  of  the  ongoing  events  in  the 
classroom  and  the  small  tenuous  bridge  we  try 
to  build  between  it  and  the  world  beyond. 


Triangulation  *:  Putting  It  All  Together 

In  rereading  my  notes  and  the  surveys, 
weeks  and  months  later,  I realize  the  differences 
in  perspectives  umu  v diiuiu  ^uxy  u w u j 

and  reportings  (the  learners’).  Right  now,  these 
surveys  tell  me  more  about  how  learners  re- 
spond to  surveys  than  about  how  they  actually 
feel  about  those  questions  posed  in  the  surveys. 
This  reminds  me  of  something  Lenore  Balliro 
reported  during  her  early  research  into  alterative 
assessment.  One  state  administrator  acknowl- 
edged openly  that  the  (standardized)  tests  being 
discussed  did  little  more  than  test  the  abilities  of 
the  test-takers  to  take  tests.  There  is  value  in  the 
process  of  asking  questions  about  how  learners 
learn  and  about  how  they  feel  they  are  pro- 
gressing. This  talk  about  learning,  however,  is 
somewhat  a language  of  its  own  and  must  be 
learned  as  (yet  another)  language. 

Learners’  responses  to  the  surveys  teach  me 
about  needing  to  find  better  ways  to  ask  the 
questions,  while  reminding  me  of  the  impor- 
tance of  ongoing  dialogue  with  learners  as  well 
as  ongoing  documentation  of  classroom  observa- 
tion. 


* “Triangulation”  refers  to  the  process  of  looking  at  various  forms  of  data  (surveys,  transcripts,  writing 
samples,  observations)  in  order  to  make  sense  of  and  derive  meaning  from  such  data.  Leo  van  Lier’s  “The 
Classroom  and  the  Language  Learners,"  (Longman,  1988)  is  an  excellent  resource  for  research  methodologies. 
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Figure  2 


What  Next? 

In  thinking  about  the  classes  to  come  (as 
this  reaches  most  of  its  readers,  new  fall  classes 
are  well  underway),  I wonder  what  might 
actually  be  the  value  of  all  this  reflection.  What 
new  goals  and  objectives  might  it  help  us  set? 
Geoff  Brindley  talks  about  puiposes,  goals  and 
objectives: 

..[SJetting  objectives  (of  whatever  kind) 
will  not  in  itself  automatically  lead  to 
more  effective  learning.  The  objectives 
might  be  irrelevant  or  incomprehensible 
to  the  learners  or  unattainable  in  terms  of 
the  resources  they  have  at  their  disposal. 

For  this  reason  it  is  very  important  that 
learners  are  involved  in  formulating  the 
objectives  and  deciding  on  the  most 
appropriate  ways  to  achieve  them.  Objec- 
tives are  not  just  a technical  tool  to  help 
teachers;  they  are  meant  to  make  learners’ 
life  easier  as  well.3 

When  I consider  what  my  learners  can  do,  I 
obviously  worry  about  what  it  is  that  they 
cannot  do.  Rosalie’s  enthusiasm  is  tempered  by 
my  own  nagging  doubts  about  what  she  and  her 
classmates  may  or  may  not  retain  over  the 
summer  and  more  importantly  about  the  con- 
nections they  make  between  the  print  related 
work  we  do  every  day  at  the  blackboard  and  the 
maze  of  print  they  confront  outside  of  school 
each  and  every  day.  Surprise’s  letter  validates  all 
of  us,  I think,  in  that  she  has  either  learned  well 
what  she  thinks  I need  to  hear  or  (I  hope)  that 
she  herself  does  see  the  possibilities  inherent  in 
those  small  but  solid  steps  she’s  taken  in  the  six 
months  since  she  started  school. 

Brindley’s  work  provides  a useful  frame- 
work for  thinking  about  concrete  ways  of 
working  with  learners  towards  involving  them 
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not  only  in  the  assessment  of  their  own  learn- 
ing but  more  importantly,  in  the  evolution  and 
shaping  of  that  learning  process.  Our  own 
reflection,  coupled  with  ongoing  investigation, 
action,  interaction,  and  dialogue  should  help  us 
further  learners’  abilities  and  options  for  and 
around  writing,  reading,  and  using  language 
and  literacy  in  ways  that  are  meaningful  to 
them. 

A Final  Note 

One  shortcoming  attributed  to  adult 
education,  particula  rly  ESL,  is  that  we  never 
seem  to  have  a sense  of  closure.  As  long  as  the 
program  is  funded,  learners  can  come  back  and 
work  within  it  for  as  long  as  they  need  to. 
Although  in  recent  years,  funding  constraints 
have  forced  us  to  shut  down  for  the  summer, 
there  isn’t  the  same  kind  of  closure  for  our 
learners  that  many  of  us  may  have  felt  at  the 
completion  of  high  school  or  college.  Learners 
come  back  to  classes  and  their  learning  contin- 
ues. It  seems  that  the  summer  hiatus  passes 
almost  unnoticed;  we  pick  up  where  we  left 
off.  When  learners  actually  leave  the  literacy 
program,  I may  feel  some  sense  of  closure,  but 
they  still  state,  “I  need  English.”  Many  of 
them  will  continue  for  as  long  as  they  can  in 
one  or  another  mainstream  ESL  program. 
Others  will  need  another  cycle  or  cycles  in 
literacy  before  feeling  ready  to  try  out  the  next 
step.  Because  of  the  ongoing  nature  of  this 
process-learners  learning  at  varying  rates,  and 
with  varying  degrees  of  speed  and  abilities  — 
our  attempts  to  reflea  upon  progress  at  the 
end  of  the  cycle  are  really  part  of  a larger, 
circular  progress  of  learning  about  learning  we 
never  really  finish.  We  lay  foundations,  we 
hope  that  learners  gain  the  needed  confidence, 
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community,  and  validation  they  will  need  to 
continue  learning.  We  need  these  things,  too. 

And  finally,  Rosalie:  “I  can  write  something, 
I’m  happy  because  somebody  can  read  and  see 
what  I’m  saying.”  That’s  a goal  right  there. 


Notes 

1.  By  “metawork,"  I refer  to  processes  of  metacognition— 

knowing  about  knowing.  It  is  largely  in  those  pro- 
cesses that  we  strive  to  engage  learners  when  working 
towards  learner-centered  assessment  and  learning. 

2.  For  a very  different  take  on  learners  evaluating  teachers, 

see  A.  Wennerstrom  and  P.  Heiser’s  “ESL  Student 
Bias  in  Instructional  Evaluation,"  TESOL  Quarterly 
(1992)  Vol  26.  No  2.  In  it,  the  authors  touch  on  cross 
cultural  differences  in  students’  perceptions  of 
teachers,  but  their  focus  is  geared  toward  performance 
evaluations  and  subsequent  career  implications  for 
those  being  evaluated.  Although  the  focus  differs 
radically  from  the  kind  of  reflective  assessment 
generally  discussed  within  this  series  (Adventures  in 
Assessment),  it  is  important  to  remain  mindful  and 
aware  of  those  “other"  forms  of  evaluation  and 
assessment;  they’re  the  “norm." 

3.  G.  Brindley  (1989).  Assessing  Achievement  in  the  Learner- 

centered  Curriculum,  Macquarie  University  National 
Center  for  English  Teaching  and  Research,  p.  9. 
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Sitting  Down  Together  at  the 

End  of  the  Year 


by  Ann  Cason 


Three  years  ago  I wrote  about  my  experi- 
ence with  all-program  evaluations  at 
Jackson  Mann  Community  School  in 
Allston.1  Jackson  Mann  is  a large  community 
center  with  a comprehensive  adult  education 
program  with  full  time  staff.  It  is  well  estab- 
lished with  a history  of  participatory  program 
management.  Currently  I am  teaching  in  a 
vastly  different  setting,  a small,  new  ESL  pro- 
gram which  is  one  of  many  programs  at  the  Log 
School  Family  Education  Center,  a settlement 
house  on  Bowdoin  Street  in  Dorchester. 

About  the  Log  School 

The  Log  School  has  been  a part  of  the 
Bowdoin  Street  neighborhood  for  over  20  years 
and  is  small  enough  so  that  community  resi- 
dents are  often  involved  in  various  programs 
within  the  center.  More  importantly,  there  is 
an  agency-wide  philosophy  of  community 
empowerment  and  strong  ties  between  the 
programs  within  the  center.  Students  who  enter 
the  ESL  program  often  have  other  connections 
within  the  Log  School.  They  may  be  volunteers 
working  in  the  Friday  morning  food  pantry  or 
parents  with  children  in  the  pre-school.  Many 
are  members  of  the  Family  Cooperative  and 
participate  in  aerobics  classes  or  the  crafts  circle 
' or  take  advantage  of  monthly  holistic  health 


clinics.  The  connections  which  ESL  students 
have  to  programs  within  the  Log  School  mean 
that  students’  roles  at  the  center  go  beyond  that 
of  ESL  students.  These  diverse  roles  mean  that 
program  participants  have  a broad  view  of  the 
program  and  the  center-a  view  that  goes  beyond 
what  happens  in  the  classroom.  It  also  means 
that  students  feel  a sense  of  ownership.  This  is 
their  community  center. 

A New  ESL  Program 

In  many  ways,  the  setting  of  the  multi- 
service center,  especially  one  which  includes  a 
family  cooperative,  is  ideal  for  a participatory 
program.  The  fact  that  the  ESL  program  is  new, 
and  that  it  was  begun  in  response  to  requests 
from  community  members,  and  that  many 
students  were  involved  with  the  center  before 
the  ESL  program  was  established,  further  suits  it 
to  participatory  program  development.  The 
teachers  and  students  meet  together  frequently, 
both  in  classes  and  at  social' events,  workshops 
and  orientations.  Students  have  many  channels 
within  the  center  to  evaluate  the  program  and 
the  classes.  Many  students  have  strong  ties  to 
the  outreach  worker  at  the  school,  others  attend 
monthly  cooperative  meetings  to  develop  and 
evaluate  activities  at  the  school.  Yet  even  in  this 
setting,  unless  a system  for  input  is  established. 
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the  hierarchical  structure  to  which  most  of  us 
are  accustomed  falls  into  place.  In  addition, 
with  the  very  limited  resources  of  our  program, 
it  is  difficult  to  find  time  for  staff  and  students 
to  meet  together.  * 


" Many  students  spoke  of  the 
difficulty  of  coming  to  classes  when 
one  is  raising  children , looking  for  a 
job,  and  going  to  appointments.  “ 


The  All-program  Evaluation 

After  a little  more  than  a year  of  ESL 
program  operation,  we  decided  to  join  the 
classes  for  all-program  evaluations.2  While  all 
teachers  work  with  their  classes  on  evaluating 
how  well  the  classes  are  working  and  how  well 
learners  are  meeting  their  goals,  our  idea  was  to 
get  beyond  curriculum  and  teaching  styles  to  an 
evaluation  of  how  the  overall-program  is  meet- 
ing the  community’s  needs. 

We  had  intended  to  meet  as  an  entire 
program,  but  instead  met  in  two  separate  groups 
to  facilitate  the  participation  of  as  many  stu- 
dents as  possible.  We,  as  teachers,  came  to  the 
evaluation  meetings  with  specific  questions 
meant  to  spark  discussion  if  needed.  These  we 
wrote  on  the  board.3  (see  Appendix  1) 

The  two  groups  of  students  and  teachers 
responded  differently.  The  first  group-the 
smaller  one- was  a group  of  about  twenty 
students.  This  group  used  the  questions  as  a 
springboard  for  the  discussion  of  the  program. 
The  question  that  we  focused  on  most  was 
retention  and  what  everyone  in  the  program  can 
do  to  enable  students  to  participate  fully  in  a 


complete  cycle.  Students  had  diverse  ideas. 

Some  suggested  that  new  students  need  a more 
complete  orientation  to  the  program  and  to  the 
whole  process  of  learning  English.  As  a woman 
from  Brazil  said,  “Oh  my  god,  the  first  day  I 
came  here,  I felt  like  a baby.  I was  so  nervous.  I 
came  with  my  husband.  I said  to  him,  ‘Tell 
them  I don’t  speak  English.’” 

Many  students  spoke  of  the  difficulty  of 
coming  to  classes  when  one  is  raising  children, 
looking  for  a job,  and  going  to  appointments. 
We,  as  teachers,  spoke  of  our  need  for  informa- 
tion about  when  the  classes  or  programs  are  not 
meeting  Students’  uceuS,  aS  well  as  the  frustra- 
tion we  face  of  turning  away  students  seeking 
classes  because  we  don’t  want  to  fill  someone’s 
slot  who  may  be  planning  to  return.  Some 
students  suggested  that  teachers  be  more  de- 
manding about  attendance. 

The  second  meeting  was  a little  bigger  and 
the  group  followed  the  questions  more  closely. 
One  student  spoke  of  the  importance  of  the 
native  language  tutoring  project  where  every  day 
for  the  past  two  cycles  a more  advanced  student 
has  been  working  with  her.  Other  students 
spoke  of  the  need  to  change  the  hours  of  the 
class  and  we  talked  about  the  possibility  of  a 
morning  class  for  the  fall.  We  discussed  the 
difficulties  of  child  care  (we  have  child  care 
available  only  during  the  afternoon)  and  of 
limited  physical  space  at  the  school. 

Some  students  talked  about  wanting  teach- 
ers to  assign  homework  every  day,  while  other 
students  said  that  they  receive  homework  every 
day  but  not  everyone  does  it.  Some  students 
suggested  that  there  needed  to  be  more  time 
spent  reviewing  homework  in  class  because 
many  students  don’t  understand  it  completely. 
One  student  said  she  tries  to  get  extra  help  as 
much  as  possible  from  the  family  advocate  at  the 
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school,  but  that  the  family  advocate  does  not 
always  have  time. 

I typed  up  the  notes  from  both  of  the 
evaluations  so  students  could  have  a record  of 
what  we  discussed  in  both  groups.  I also  typed 
a letter  to  students  with  the  program  changes 
that  we  were  implementing  along  with  some  of 
the  changes  we  were  not  able  to  make  at  this 
time,  but  that  we  are  exploring  for  the  future. 

Changes 

These  discussions  affected  not  only  the 
teachers’  methods,  but  the  program  as  a whole. 
Here  are  a few  of  the  concrete  changes  that  came 
about  as  a result  of  the  all-program  evaluations: 

• We  decided  to  establish  two  drop-in 
tutoring  times,  one  before  the  first 
class  in  the  afternoon,  and  one  later  in 
the  afternoon  to  enable  students  to  get 
some  extra  help  on  homework  or 
other  learning  areas. 

• Through  discussion  of  the  balance 
between  reasonable  expectations  and 
the  logistics  of  dealing  with  life  and 
the  need  to  consider  students  on  our 
waiting  list,  we  decided  upon  a clearer 
orientation  on  program  objectives 
before  classes  begin,  and  to  include 
sharing  by  students  who  have  been 
involved  in  the  program  for  some 
time. 

• Resulting  from  the  second  meeting, 
we  are  considering  a morning  class  for 
September,  depending  on  space  and 
the  needs  of  incoming  and  current 
students. 

• We  decided  to  expand  our  participa- 
tory philosophy  to  include  the  orien- 


tation process  so  that  students  are 
aware,  from  the  beginning  of  their 
connection  to  the  program,  that  we 
have  strong  beliefs  about  teaching  and 
operating  a program. 

• We  instituted  a new  attendance  policy. 
If  a student  misses  three  classes,  a 
teacher  will  contact  him  or  her  to  see 
if  the  student  plans  to  return  to  the 
program  that  cycle. 

Concrete  changes  are  important.  It  is  en- 
couraging when  there  are  some  straightforward 
solutions  or  compromises  to  try  out.  Often, 
however,  teachers  struggle  with  the  lack  of 
funds  and  other  resources  and  feel  there  are  few 
options,  not  seeing  the  value  in  exploring  what 
students  would  ideally  like  from  the  program. 
While  the  immediate  changes  that  we  can  make 
are  the  most  concrete  benefit  of  exploring  these 
needs,  they  are  not  the  only,  nor  perhaps  the 
most  important  ones.  The  less  tangible  gains  of 
building  the  program’s  sense  of  community, 

These  discussions  affected  not  only 
the  teachers’  methods , but  the 
program  as  a whole. 


envisioning  the  program  as  part  of  a larger 
entity,  and  getting  outside  the  classroom  to 
involve  students  in  leadership  roles  may  be  far 
more  important  in  the  long  run. 

It  is  the  discussion,  the  opportunity  not 
only  for  student  feedback,  but  for  dialogue 
across  classes  and  between  students  and  teachers 
which  is  most  important  in  this  enterprise.  I 
believe  that  the  more  we  share  with  students  our 
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vision  of  the  ESL  program  as  one  component  of 
a larger  whole  and  as  a community  program 
which  is  meant  to  respond  to  their  interests  and 
needs,  the  broader  the  effects  of  the  program, 
and  the  way  in  which  students  view  the  pro- 
gram, will  in  turn  have  an  effect  upon  classroom 
learning,  curriculum,  and  attendance. 

As  students  see  the  ESL  program  as  pan  of 
a larger  whole,  they  begin  to  bring  more  com- 
munity issues  and  concerns  to  the  program,  and 
to  see  those  concerns  as  legitimate  topics  of 
discussion  and  action  for  the  ESL  class.  One 
example  of  this  in  our  program  has  been  the 
issue  of  safety  in  Ronan  Park,  a nearby  park. 
The  park  is  beautiful,  has  a playground,  basket- 
ball couns,  and  overlooks  the  harbor,  but  is 
underutilized  due  to  safety  concerns.  A few 
weeks  after  the  all-program  evaluation,  when  we 
were  planning  a picnic  in  Ronan  Park,  students 
in  one  class  brought  their  concerns  about  the 
park  to  class  and  asked  the  teachers  to  set  up  a 
meeting  with  the  Log  School  director  and  the 
police.  Since  this  time,  several  students  have 
participated  in  meetings  on  neighborhood  safety 
with  a coalition  of  community  residents,  mer- 
chant, health,  and  social  service  organizations, 
and  the  police.  One  literacy  class  had  a discus- 
sion and  language  experience  letter  centered 
around  the  strategy  of  community  crime  watch. 
Police  feel  that  this  is  the  most  effective  tactic, 
while  students  are  frightened  of  repercussions 
and  feel  that  the  police  need  to  take  more  re- 
sponsibility. 

We  still  have  a long  way  to  go  in  terms  of 
developing  our  program  evaluation  meetings. 
We  still  struggle  with  lack  of  time  in  everyone’s 
schedules  to  reflea  among  ourselves  as  teachers. 
I think  at  the  next  program  evaluation  it  would 
be  good  to  ask  students  to  develop  questions 


before  the  evaluation.  While  I have  learned 
enormously  from  students,  volunteers,  and  staff, 
both  at  the  Jackson  Mann  and  at  the  Log 
School,  I feel  as  though  I still  have  much  more 
to  learn.  I think  there  is  sometimes  a push  in 
adult  education  for  new  and  exciting  tools,  so  at 
times  we  underestimate  the  value  of  something 
which  is  so  simple. 

Experiences  at  both  Jackson  Mann  and  the 
Log  School  have  taught  me  that  the  all-program 
evaluation  elicits  information  and  builds  com- 
munity. It  is  a tool  that,  while  helping  vis  to 
look  back,  is  really  just  a beginning. 


Notes 

1.  Nash,  A.,  Cason,  A.,  Gomez-Sanford,  R.,  McGrail,  L., 

and  Rhum,  M.  (in  press).  Talking  Shop:  A Curricu- 
lum Sourcebook  for  Participatory  Adult  ESL. 
Englewood  Cliffs,  NJ:  Regents/Prentice  Hall. 

2.  It  is  important  to  note  that  this  all-program  evaluation 

took  the  form  of  a town  meeting.  It  was  not 
intended  as  an  objective  tool  for  producing  quantifi- 
able data  on  how  the  program  operates.  Rather,  the 
point  was  to  receive  feedback  on  program  operation 
and  share  information  about  and  responsibility  for 
program  operation. 

3.  Program  participants  at  the  Log  School  are  from  Cape 

Verde,  Haiti,  Latin  America  and  the  Caribbean  and 
Vietnam.  The  first  group  did  not  require  translation 
because  of  their  more  advanced  English  skills.  For 
the  second  group,  translation  was  provided  by 
advanced  students  from  the  first  group  and  a teacher 
who  speaks  Haitian  Creole. 
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Program  Evaluation  at  the 
Community  Learning  Center 


by  Mina  Reddy 


he  Community  Learning  Center  is  the 
adult  basic  education  center  of  the  City 
of  Cambridge.  We  have  classes  in  English 
as  a second  language,  basic  reading,  writing,  and 
math,  GED  preparation,  and  an  Adult  Diploma 
Program.  Over  1000  students  a year  come  to  the 
program,  60%  of  them  for  ESL.  The  Learning 
Center  has  18  salaried  staff  members,  most  of 


“ Staff  sometimes  had  difficulty  seeing 
the  difference  between  making  a plan 
to  solve  a problem  and  solving  the 
problem  on  the  spot. " 


them  full  time,  and  29  part  time  staff.  Salaried 
staff  (teachers,  counselors,  and  administrative 
staff)  meet  weekly  in  staff  or  departmental 
meetings.  The  CLC  operates  with  a group 
decision-making  process.  All  major  decisions 
must  be  made  by  majority  vote  of  the  entire 
salaried  staff.  Each  department  (administration, 
ESL,  and  ABE/high  school)  selects  a coordina- 
tor. The  three  department  coordinators  and  the 
director  meet  every  other  week.  They  can  make 
some  limited  decisions  but  are  primarily  respon- 


sible for  maintaining  good  com  muni  cation 
between  and  within  departments  and  structur- 
ing center-wide  activities  like  evaluations.  Two 
student  advisory  committees,  one  for  morning 
and  one  for  evening  students,  and  made  up  of 
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one  representative  from  each  class,  meet 
monthly  to  make  suggestions  to  the  staff  and 
help  organize  activities  like  parties  and 
fundraisers.  Discussions  led  in  class  by  these 
members  help  give  students  an  overall  picture  of 
the  school  and  make  them  feel  better  able  to 
evaluate  it.  Informal  program  evaluation  occurs 
in  staff,  department,  coordinator,  and  student 
advisory  council  meetings  all  year.  A more 
formal  program  evaluation  process  occurs 
yearly.  Questionnaires  are  distributed  to  staff 
and  students  and  meetings  are  held  to  discuss  the 
results  and  plan  program  development  for  the 
following  year,  (see  Fig.  1 and  Appendix  2) 


Development  of  the  Staff 
Evaluation  Form 

In  previous  years,  we  had  used  a brief  open- 
ended  form  for  staff  program  evaluation.  This 
year,  in  part  because  we  had  made  a commit- 
ment, for  the  first  time,  to  develop  a three-year 
plan,  we  decided  to  do  a more  extensive  process. 
Although  we  had  not  joined  the  SABES  pro- 
gram development  pilot,  we  asked  for  a copy  of 
the  materials  distributed  at  the  training  for  the 
pilot.  We  based  our  form  on  the  “Indicators  of 
Program  Quality  Checklist”  developed  by 
SABES  and  spent  two  coordinators  meetings 
modifying  it  to  fit  our  program. 

Administration  of  the  Staff 
Evaluation  Form 

The  form  was  explained  in  a staff  meeting 
and  was  distributed  to  all  full  and  part  time  staff. 
They  were  told  they  could  skip  any  questions 
about  which  they  did  not  have  enough  informa- 
tion, and  they  were  asked  to  star  their  top  five 
priorities  among  needs.  Part  time  teachers  were 
given  an  additional  hour  of  paid  time  to  fill  out 
the  form.  People  took  it  very  seriously  and,  in 
some  cases,  wrote  extensive  comments;  they 
appreciated  being  asked  their  opinions. 

The  results  were  tallied  separately  for  full 
and  part  rime  staff  and  then  combined  and  all 
comments  (15  pages  of  them)  were  typed  up. 
Tallies  and  comments  were  distributed  to  all 
staff. 

Development  of  the  Student 
Evaluation  Form 

In  February  1991,  in  morning  and  evening 
student  advisory  committee  meetings,  students 
were  asked  “What  do  you  want  to  learn  in 
school?”  and  “Does  the  school  help  you  with 
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anything  you  do  outside?"  Students  came  up 
with  a lot  of  specific  answers,  including  some 
that  staff  might  not  have  thought  of  putting  on 
a questionnaire  like,  “Help  me  enjoy  reading,” 
and  “Be  able  to  make  friends.”  These  ideas, 
along  with  a few  additional  ones  added  by 
teachers,  were  turned  into  questions  on  a check- 
list for  students  to  fill  out  in  a program  evalua- 
tion questionnaire.  Two  problems  arose,  how- 
ever, when  the  questionnaire  was  administered 
in  1991.  The  checklist  was  a long  list  of  items 
with  directions  at  the  top  asking  students  to 
check  what  the  school  had  helped  them  with.  In 
some  cases,  students  forgot  or  misunderstood 
the  directions  and  checked  everything  they  were 
able  to  do  even  if  the  school  had  nothing  to  do 
with  it.  Also,  students  who  did  not  have  a job 
were  checking  that  the  school  had  helped  them 
find  a job  and  students  who  did  not  take  math 
responded  to  some  of  the  questions  specifically 
related  to  activities  in  the  math  classes.  The 
form  was  therefore  revised  last  spring  by  break- 
ing it  up  into  sections,  making  the  questions 
clearer,  and  allowing  students  to  skip  sections 
that  were  not  relevant.  The  revision  was  based 
on  feedback  from  teachers  and  a meeting  be- 
tween the  director  and  two  representatives  of 
the  student  advisory  committee,  (see  Fig.  2 and 
Appendix  3) 

Administering  the  Student  Program 
Evaluation  Form 

Copies  of  the  form  were  given  to  all  teach- 
ers to  distribute  and  explain  to  students  in  late 
May.  Teachers  of  beginning  ESL  students  could 
choose  not  to  use  it  if  it  would  be  too  difficult 
for  students  to  understand.  We  received  321 
responses.  We  were  fortunate  to  have  a VISTA 


Comments  from  the  student 
questionnaires 

• Please  give  other  examples  of  what 
you  learned  in  school  and  how  the 
school  has  helped  you. 

", Before  1 came  here  I didn  ‘t  speak  English. 
I remember  once  1 went  to  the  hospital.  I 
couldn't  find  an  interpreter.  1 came  back 
without  an  examination.  Now  1 can 
explain  everything  1 want.  * 

❖ 

"/  really  learned  in  school,  to  lose  the 
timidity,  to  communicate,  to  read  a 
little.B 

❖ 

“This  school  gave  me  a second  chance  to 
succeed  in  life.  1 made  a mistake  before  in 
not  finishing  school  and  this  school  gave 
me  hope  for  a better  future.  * 

❖ 

“1  learn  to  have  an  open  mind  by  coming 
to  school,  lean  think  better. * 

❖ 

“The  school  has  given  me  the  understand- 
ing  of  other  people  and  their  cultures,  of 
how  diversified  we  all  are,  and  that  we 
have  one  common  interest— to  better 
ourselves.  * 

❖ 

“Since  I came  back  to  the  Learning 
Center  it  has  helped  me  a lot  My  English 
comes  better  every  day.  I had  a promotion 
in  my  job.  * 
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volunteer  to  tally  the  responses,  turn  them  into 
percentages,  and  type  the  comments— 32  pages 
of  them! 


Analyzing  the  Data 

Coordinators  met  to  summarize  the  staff 
program  evaluations  and  decide  on  the  areas  of 
greatest  need  for  development.  In  order  to 
determine  the  highest  priorities,  we  considered 
1)  the  number  of  people  who  checked  an  item  as 
a need,  2)  the  number  of  stars  put  next  to  an 
item,  and  3)  comments.  (Ideally,  student  re- 
sponses would  have  been  part  of  the  discussion 
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yet  been  completed.  They  were  available  by  the 
time  action  groups  met.)  The  needs  were 
grouped  into  six  categories  which  were  revised 
in  the  next  staff  meeting.  The  final  categories 
were  curriculum,  counseling  and  follow-up, 
physical  plant,  part  timer  needs,  staff  develop- 
ment, funding,  and  assessment. 


Action  Groups 

Two  weeks  had  been  set  aside  at  the  end  of 
June  in  order  to  have  four  full  days  for  planning 
and  still  leave  enough  time  for  our  graduation 
and  several  staff  development  workshops.  In  a 
staff  meeting  the  Friday  before  the  first  planning 
week,  1)  the  coordinators  reported  on  the 
proposed  action  groups  and  specific  issues  or 
needs  to  be  worked  on  in  each  one  according  to 
staff  and  student  responses,  2)  staff  modified  and 
combined  the  groups,  and  3)  each  person  chose 
one  aiction  group  to  participate  in.  Specific  needs 
were  spelled  out  within  each  category,  e.g., 
under  curriculum:  math,  Adult  Diploma  Pro- 
gram history,  intermediate  reading,  and  work- 
place education  curriculum;  student  input  into 
curriculum;  use  of  textbooks;  use  of  technology; 
number  of  hours  of  class  per  week.  The  groups 


were  to  meet  Monday  and  Tuesday  of  the  first 
planning  week  to  draw  up  three-year  plans  for 
that  area.  Action  plan  meetings  would  be  alter- 
nated with  staff  and  departmental  meetings.  The 
following  week,  there  would  be  two  more  all- 
day staff  meetings  to  finalize  the  plan,  with 
action  groups  meeting  separately  as  needed. 

Each  action  group  was  given  a form  (see  Appen- 
dix 4),  lifted  from  a request  for  proposals,  to 
write  their  objectives,  activities,  persons  respon- 
sible, and  timeline.  Staff  sometimes  had  diffi- 
culty seeing  the  difference  between  making  a 
plan  to  solve  a problem  and  solving  the  problem 
Cu  the  Spot.  Controversy  arose  when  the  assess- 
ment group  wrote  into  the  plan  specific  tests  to 
be  given  when  the  staff  had  not  agreed  on  the 
appropriateness  of  those  tests  for  large  numbers 
of  students.  The  plan  was  revised  to  be  more 
general  and  allow  time  to  examine  and  select 
tests  and  set  policy. 

It  was  also  difficult  for  staff  to  think  in 
terms  of  a three-year  plan.  In  part  because  of  the 
nature  of  our  funding,  we  have  been  used  to 
thinking  only  one  year  at  a time.  We  tended  to 
set  September  1992  as  the  time  to  accomplish  an 
unrealistic  proportion  of  the  tasks  and  had  to 
revise  the  timelines  when  we  looked  at  all  the 
plans  together.  Before  ending  the  meetings,  we 
set  up  times  to  monitor  our  progress  towards 
our  objectives  over  the  next  year.  June  was  a 
difficult  time  to  hold  a series  of  day-long  meet- 
ings because  everyone  was  tired  from  the  semes- 
ter of  teaching  that  had  just  ended.  However,  at 
the  end  of  the  process,  we  felt  we  had  accom- 
plished something  significant. 

Next  Steps 

Once  the  meetings  were  finished,  the  task 
remained  of  typing  up  the  separate  action  plans 
into  one  document  and  prefacing  it  with  a 
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statement  of  strengths  that  were  identified  in  the 
process.  Most  responses  to  both  evaluations 
were  very  positive,  and  this  was  not  acknowl- 
edged anywhere  in  this  process  which  focused 
on  needs  and  areas  for  development.  This  work 
will  be  done  during  the  summer.  When  we  meet 
again  as  a whole  staff  in  September,  we  can 
make  any  revisions  that  seem  appropriate  at  that 
time. 

What  Will  We  Do  Next  Year? 

Will  we  use  the  same  form  and  process  next 
year?  Probably  not.  However,  we  hope  that  this 
description  of  what  we  did  is  useful  to  other 
programs  in  developing  their  own  system  of 
program  evaluation. 
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Learner-Friendly  Assessment: 

A Workplace  Model 

by  Joyce  Jackson  and  Ruth  Schwendeman 


In  late  spring  of  1991,  a central  Massachu- 
setts manufacturing  company  and 
Worcester’s  Quinsigamond  Community 
College  (QCC)  began  to  plan  a workplace 
education  program  that  would  serve  a large 
population  of  limited  English  speakers,  many  of 
whom  had  been  at  the  company  ten  years  or 
more.  The  company  wanted  a program  that 
would  not  only  serve  a training  need  but  would 
also  be  seen  as  one  of  the  many  benefits  avail- 
able to  its  employees.  It  was  paramount  that  the 
classes  be  offered  in  a non-threatening  way-that 
these  valued  workers  would  understand  there 
was  absolutely  no  risk  of  loss  of  jobs  attached  to 
their  participation. 

In  designing  the  assessment  and  placement 
process,  we  sought  to  maintain  and  expand  this 
goal.  We  believed  that  learner  input  from  the 
inception  of  the  program  was  essential  for 
building  a climate  of  openness  and  trust  desired 
by  the  company.  We  felt  that  standardized  tests 
would  not,  with  any  validity,  address  either  the 
academic  or  affective  skills  of  the  population. 

As  in  the  alternative  assessment  work  of  Elsa 
Auerbach  and  Susan  Lytle,  we  sought  to  develop 
an  intake  process  that  would  be  participant 
friendly  and  which  would  yield  information 
about  learner  abilities,  interests,  and  uses  of 
literacy  in  day-to-day  life. 


Climate  Building 

Two  pre-assessment  activities  helped  to  set 
the  stage  for  the  individual  assessment  process. 

A task  force  made  up  of  instructional  staff  and 
respected  non-supervisory  workers  (limited 
English  speakers  as  well  as  native  English  speak- 
ers) was  briefed  on  the  proposed  new  classes.  As 


“We  believed  that  learner  input 
from  the  inception  of  the  program 
was  essential  for  building  a 
climate  of  openness  and  trust 
desired  by  the  company.  * 


a means  of  recruitment,  members  of  the  task 
force  were  asked  to  share  this  information  with 
their  co-workers.  Interested  parties  could  then 
respond  by  signing  up  for  the  second  activity, 
an  informational  group  meeting  where  more 
details  about  the  program  would  be  presented. 
We  expected  around  50  workers  to  respond,  but 
nearly  90  people  signed  up! 

Fifteen  group  meetings  of  approximately 
six  employees  and  a QCC  representative  were 
conducted,  each  lasting  30  minutes.  At  these 
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’Task  Force ‘ 
Meeting 


Croup  meetings  with 
QCC  rep... 

■ get  info  about  classes 
• give  ideas  about  what 
you  want  to  learn 


Schedule  and  register 
for  classes 


Individual  meetings 

■ informal  conversation  to 
learn  your  specific  needs 


- QCC  and  company  plan  and  develop  classes 
based  on  interview  data 


/ 


Figure  1 


meetings,  we  explained  the  company’s  involve- 
ment with  QCC  and  outlined  the  remaining 
interview  and  planning  steps.  We  asked  each 
group  what  they  wanted  to  learn  and  what  they 
thought  their  educational  needs  were.  It  was 
our  intention  to  create  a relaxed  environment 
and  to  alleviate  any  fears  or  reservations  em- 
ployees may  have  had.  We  reassured  groups 
that  this  educational  program  was  entirely 
voluntary,  was  an  additional  benefit  being 
offered  by  the  company,  and  would  not  have  an 
impact  on  jobs  in  any  negative  way.  General 
information  about  the  nature  of  the  classes  was 
given  and  sign-up  forms  were  made  available  for 
individual  assessment  interviews.  The  schematic 
above  was  used  in  large  form  as  a visual  during 
the  meetings,  and  was  reduced  and  given  as  a 
handout  so  that  the  employees  could  think 
privately  about  the  process  and  make  a decision 
about  participation,  (see  Fig.  1) 

Though  the  one-on-one  meeting  was  to  be 


the  major  assessment  activity,  along  the  way  in 
the  task  force  and  group  meetings  we  were  able 
to  informally  assess  the  population.  What  was 
especially  useful  from  the  meetings  was  the 
input  we  gained  from  the  employees  about  their 
language  goals  and  the  kinds  of  language  skills 
required  by  their  jobs  and  home  lives.  Also,  we 
began  to  get  to  know  each  other,  which  was 
helpful  once  the  interviews  got  underway. 
Interviewees  were  relaxed  because  they  saw  a 
familiar  face  across  the  table. 

The  Interview 

The  interview  followed  a specified  format, 
and  used  standardized  intake  forms  (see  Fig.  2 
and  Appendix  5).  We  made  certain,  however, 
that  the  workers  understood  what  we  planned 
to  ask  and  how  the  information  would  be  used. 
We  tried  to  write  as  little  as  possible,  attempting 
instead  to  make  the  event  more  of  a conversa- 
tion, with  the  participants  identifying  their  own 
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instead  to  make  the  event  more  of  a conversa- 
tion, with  the  participants  identifying  their  own 
goals  for  language  improvement.  Questions 
focused  on  the  employees’  educational  back- 
ground, as  well  as  on  their  current  literacy 
activities  at  home  and  at  work. 

To  assess  reading  skills,  we  offered  a selec- 
tion of  materials,  allowing  readers  to  choose  any 
piece  they  felt  most  comfortable  reading  silently 
or  aloud.  We  followed  with  a brief  discussion 
to  check  comprehension.  If  the  participants  so 
chose,  a second  selection  could  be  made.  (The 
readability  levels  of  the  selections  had  been  pre- 
established,  though  no  such  identification 
appeared  on  the  selections.) 

To  assess  writing  skills  the  workers  were 
first  asked  to  fill  in  a simple  form  requiring 
name,  address,  and  other  basic  information.  A 
second  writing  task  asked  the  workers  to  com- 
pose a brief  paragraph  in  response  to  a variety  of 
possible  writing  topics,  i.e.,  a note  to  their 
child’s  teacher,  a phone  message,  etc.  If  the 
writers  felt  unable  to  complete  this  task,  they 
were  not  required  to  attempt  it.  To  maintain 
the  non-threatening  atmosphere  and  to  preserve 
the  learners’  self  esteem,  occasionally  the  inter- 
viewers dictated  a simple  sentence  if  the  learners 
were  unable  to  generate  one  alone.  In  some 
cases,  it  was  helpful  for  the  interviewers  to  leave 
the  room  briefly  while  the  interviewees  wrote, 
making  the  writers  feel  less  conscious  about 
being  “watched*  while  composing. 

A final  but  essential  step  was  to  discuss  the 
worker’s  placement  in  a class.  Given  the  possi- 
bility of  three  ESL  levels,  the  learners  were 

where  they  would  feel  most  comfortable. 
This  interchange  allowed  the  learners  to  be  part 
of  their  own  assessment,  to  take  stock  of  their 
own  skills.  The  interviewers  also  offered  input 
on  the  subject,  but  made  sure  the  final  determi- 


nation reflected  the  judgement  of  both  parties. 

Once  the  interview  was  complete,  we  again 
advised  learners  of  when  the  program  would 
begin  and  answered  any  questions  that  arose. 
Using  the  guideline,  “listen  now  and  write 
later,”  the  interviewers  determined  1)  the  skills 
students  already  had,  2)  target  areas  for  skills 
improvement,  3)  problem  areas,  4)  student- 
identified  objectives,  and  5)  a mutually  agreed 
upon  class  level. 

Results 

We  shared  all  the  interview  information 
with  teachers  in  the  program,  and  their  curricu- 
lum planning  mirrored  the  ability  levels  deter- 
mined in  the  interviews. 

This  assessment  procedure  met  our  goals  of 
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and  tools  have  since  been  used  at  two  other 
sites,  and  we  are  working  to  replicate  the  pro- 
cess throughout  Quinsigamond’s  Workplace 
Education  program.  At  one  location,  a large 
high-tech  company,  the  procedure  was  em- 
ployed under  somewhat  different  circumstances. 
We  used  the  alternative  assessment  format  to 
supplement  standardized  placement  test  infor- 
mation previously  gathered.  In  doing  so,  we 
were  able  to  expand  our  picture  of  learner 
abilities  and  needs  and  to  involve  learners  in  self- 
assessment  and  the  goal  setting  process. 

At  another  site,  a medium-sized  tool  manu- 
facturing company,  the  format  was  used  to 
determine  the  skills  of  a multi-leveled,  native 
and  non-native  English  speaking  population. 
Information  gained  from  the  assessment  process 
was  then  .used  to  develop  the  pilot  workplace 
education  class  at  the  company.  In  this  instance, 
the  assessment  process  provided  a good  generic 
tool  for  assessing  a wide  range  of  learner  needs 
and  abilities  both  ESL  and  ABE.  Participants  at 
this  company  came  into  the  interview  with  a 
fair  amount  of  anxiety  about  “going  to  school,” 
but  appeared  to  leave  the  assessment  in  a more 
relaxed  state  and  with  a positive  attitude  toward 
the  program. 

At  the  original  location,  we  were  fortunate 
to  have  many  activities  where  learner  input 
could  be  elicited  and  where  informal  assessment 
could  take  place.  We  learned  that  providing 
several  opportunities  for  learner  “buy-in,” 
strengthened  the  participant’s  commitment  to 
the  program  from  inception  to  completion. 
Given  the  constraints  of  individual  companies 
to  allow  for  generous  allocation  of  employee 
release  time  to  attend  several  meetings,  we 
realize  that  is  not  always  possible.  Even  with- 
out such  substantial  initial  release  time,  we  did 
find  that  this  basic  alternative  assessment  format 


proved  to  be  viseful  and  adaptable.  We  do  not 
say  that  it  offers  hard,  quantitative  data.  That 
was  not  our  goal.  We  felt  that  affective  needs  as 
well  as  academic  skills  should  be  addressed  and 
that  this  procedure  allowed  us  to  include  those 
elements  in  an  initial  evaluation  process. 

The  assessment  is  learner  friendly.  It 
relieves  much  of  the  anxiety  learners  feel  when 
they  hear  the  word  “assessment.”  We  are  still 
fine-tuning  the  process.  We’d  like  to  get  beyond 
the  stage  where  all  ESL  learners  say,  “I  need  to 
learn  to  spell,”  to  a situation  where  they/we  can 
better  identify  their  strengths  and  needs.  We 
want  also  to  begin  work  on  interim  progress 
assessment  tools  that  bear  the  same  characteris- 
tics as  the  initial  assessment  format,  helping 
learners  to  clearly  articulate  what  they  have 
learned,  and  what  still  needs  work.  The  alterna- 
tive assessment  process  is  preparation  for  the 
learner-centered,  cooperative  learning  and  self- 
assessment  model  of  instruction  currently  in  use 
at  QCC. 
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Assessment  and  Planning: 
Giving  Students  Ownership 

Amy  Gluckman,  Jeff  Ritter,  Anne  Mullen  and  Kathy  Lento 


Just  A Start  is  a comprehensive  community 
service  organization.  Each  part  of  Just  A 
Stan  has  an  educational  component.  One 
Stop  is  for  out-of-school  young  people,  aged  16- 
21,  who  are  interested  in  job  readiness,  carpen- 
try and  painting  skills  training,  and  education. 
Futures  for  Young  Parents  involves  pregnant 
parenting  teens  in  GED  preparation  and 
career  exploration.  The  Evening  School  meets 
three  nights  a week  and  is  open  to  all  adults  in 
the  community  who  want  to  work  toward 
taking  their  GED  tests.  Each  of  these  programs 
strives  toward  having  individualized  learning 
programs  and  a student-centered  curriculum. 

Many  of  our  students  are  not  ready  to  begin 
GED  preparation.  These  students  are  placed  in 
pre-GED  classes,  and  it  will  be  many  months  or 
perhaps  years  before  they  will  be  able  to  pass 
the  GED.  For  this  reason,  last  year  we  began  to 
explore  ways  in  which  these  students  could  see 
how  they  were  making  progress  toward  their 
long-term  goal,  and  how  they  were  accomplish- 
ing short-term  goals.  We  began  looking  at 
various  instruments  that  would  allow  the  stu- 
dents to  set  their  own  goals  and  become  the 
planners  and  owners  of  their  accomplishments. 
Teachers  would  provide  the  tools,  but  the 
students  would  be  responsible  for  using  them  in 
a way  that  best  met  their  needs.  Both  teachers 
and  students  would  gain  some  freedom  from 


complete  reliance  on  the  standardized  testing 
required  by  our  funders. 

The  development  process  took  many 
months.  SABES  provided  descriptions  of  the 
student-centered  goal  setting  and  assessment 
procedures  that  other  Massachusetts  programs 
had  developed,  along  with  copies  of  the  instru- 
ments they  had  created.  Using  these  as  a model, 
we  developed  an  intake  and  ongoing  assessment 
process  that  would  be  more  student-centered 
than  our  current  approach. 


The  Process 

The  assessment  procedure  we  developed 
includes  an  extensive  intake  process  along  with 
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periodic  review  of  individual  student’s  goals  and 
progress  in  one-on-one  meetings. 

Intake  Assessment:  Some  of  our  programs 
are  open-entry/open-exit  while  others  have 
definite  cycles.  In  either  case,  the  teacher  and  the 
new  student  sit  down  together  and  fill  out  the 
Intake  Registration  form,  (see  Fig.  1-2  and 
Appendix  6)  This  form  asks  students  to  rate 
their  own  skills  in  reading,  writing,  and  math.  It 
also  asks  about  their  educational  history.  Next, 
students  are  asked  to  write  a paragraph  on  any 
topic  to  give  the  teacher  an  idea  of  the  student’s 
skill  in  writing  and  to  provide  a sample  to 
which  later  writings  can  be  compared. 

Within  the  first  week  of  class,  the  students 
fill  out  the  Student  Learning  Goals  form  (see 
Fig.  3 and  Appendix  7)  This  eight-page  form 
lists  a wide  range  of  skills,  subjects  and  practical 
activities  that  students  might  want  to  do  or 


learn.  Finally,  new  students  take  a standardized 
test;  over  the  past  year  we  have  used  both  the 
SRA  and  the  TABE. 

Once  the  student  has  been  in  class  for  two 
or  three  weeks,  the  student  and  teacher  have  a 
meeting.  At  this  time,  we  look  over  the 
student’s  goals,  interests,  test  results,  and  any 
other  information.  Together,  we  make  a plan 
for  the  next  eight  to  twelve  weeks  using  the 
Educational  Goals  Plan  and  Progress  Record. 

(see  Fig.  4 and  Appendix  8)  The  Goals  Plan 
describes  what  each  student  will  be  working  on 
over  the  next  few  months  and  where  each  hopes 
to  be  at  the  end  of  that  time. 

Ongoing  Assessment:  As  students  con- 
tinue in  class,  the  teacher  and  the  students  use 
the  information  on  the  Goals  Plan  to  develop 
weekly  or  biweekly  assignment  sheets  which  are 
kept  in  the  students’  notebooks.  In  order  to 
keep  track  of  the  work  students  have  done,  they 
fill  out  the  daily  log1  at  the  end  of  each  day; 
students  keep  these  logs  in  their  notebooks  as 
well.  Then,  after  two  or  three  months  in  the 
class,  the  teacher  meets  with  students  individu- 
ally. At  this  meeting,  the  Goals  Plan  is  reviewed 
and  updated,  and  a new  plan  for  the  next  period 
is  written.  The  purpose  of  these  periodic  meet- 
ings is  to  give  students  an  opportunity  to  appre- 
ciate what  they  have  accomplished,  to  discuss 
any  problems  that  the  student  or  the  teacher 
may  be  having,  and  to  plan  goals  and  work  for 
the  next  few  months  (see  Fig.  5 and  Appendix  9). 

Looking  Back 

This  description  of  the  assessment  pro- 
cess we  developed  summarizes  what  we  planned 
to  do  more  accurately  than  what  we  actually 
did.  In  practice,  different  teachers  have  found 
different  parts  of  the  assessment  plan  to  be  more 
or  less  useful  and  more  or  less  feasible.  Below, 
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three  of  the  teachers  discuss  their  experiences 
with  the  process  and  how  the  tools  worked  in 
practice. 

JEFF:  The  One  Stop  Program 

My  overall  experience  with  the  assessment 
methods  we  developed  at  Just  A Start  has  been 
positive.  The  Daily  Log  has  been  a great  help.  I 
can  see  the  students’  amazement  at  how  much 
work  they  have  done  in  a day.  They  also  express 
some  dismay  if  they  realize  that  they  have  not 
produced  much.  The  Student  Learning  Goals  is 
always  helpful  as  a reference  for  both  me  and 
the  students  when  they  need  to  reignite  their 
motivation  or  remind  themselves  of  something 
they  committed  to  earlier.  The  weekly  Assign- 
ment Sheets  have  been  the  least  useful.  The 
week  is  really  an  unknown  quantity  on  a Mon- 
day and  there  is  no  reason  to  expect  or  want 
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everything  on  a lesson  plan  to  get  accomplished. 
Most  weeks,  something  unexpea ed  occurs  that 
precludes  finishing  the  assignments.  Spontane- 
ous lessons  usually  turn  out  to  be  better  learning 
experiences. 

KATHY:  The  Futures  for  Young 
Parents  Program 

Of  all  the  materials  generated  by  this 
projea,  I found  the  Student  Learning  Goals  list 
to  be  the  least  helpful.  The  form  itself  is  confus- 
ing; it  is  long  and  a bit  repetitive.  The  students 
are  not  always  sure  how  to  respond  and  don’t 
feel  it  is  helpful  either.  I would  like  for  us  to 
develop  something  shorter,  that  is  filled  out 
with  the  student.  I think  the  Daily  Log  has  the 
potential  to  work  well  and  be  useful  for  many 
things.  The  problem  is  in  reminding  the  stu- 
dents to  fill  it  out  and  in  giving  them  feedback 
on  what  they  have  written.  The  Weekly  Assign- 
ment Sheet  is  good,  although  it  could  be  more 
closely  connected  to  the  daily  log.  We  are  trying 
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to  devise  a system  where  the  sheets  are  more 
accessible  to  teachers  so  that  they  are  actually 
filled  out  in  a timely  way.  The  Goals  Plan 
works  the  best.  It  takes  a long  time  for  students 
to  really  feel  that  they  want  to  be  involved  in 
setting  goals  for  themselves. 

AMY:  The  Futures  for  Young 
Parents  Program 

Overall,  I rhink  that  the  process  of  thinking 
about  setting  and  assessing  goals  in  a student- 
centered  way  has  been  helpful  to  me  as  a 
teacher.  Ironically,  it  has  probably  forced  me  to 
acknowledge  that  I don’t  believe  as  completely 
in  a student-centered  approach  as  I might  have 
thought-that  there  are  some  things  that  I want 


the  students  to  learn  and  some  goals  that  I want 
them  to  have  whatever  they  think  about  it! 

The  process  and  instruments  that  we  are 
now  (trying  to)  use  have,  I think,  been  helpful 
to  the  students,  although  it  is  hard  always  to 
know  how  and  how  much.  The  Goals  Plan, 
along  with  the  one-on-one  meetings  at  which  it 
is  filled  out,  is  the  most  important  to  my  mind. 
We  have  had  a hard  time  scheduling  these  one- 
on-one  meetings,  so  they  have  not  been  held  as 
regularly  as  we  planned.  (W e are  planning  to 
build  time  slots  for  them  into  our  1992-93 
schedule.)  Still,  when  a student  seemed  to  feel 
discouraged  about  progress,  pulling  out  the 
Goals  Plan  and  reviewing  it  was  a perfect  thing 
to  do.  T iding  everything  that  the  student  had 
accomplished  in  the  previous  two  or  three 
months  helped  the  student  feel  better  about 
being  in  school,  and  planning  out  the  next  few 
months  made  the  path  toward  getting  a GED 
seem  finite. 

The  Daily  Log  is  also  helpful,  primarily  in 
helping  the  students  stay  in  touch  with  what 
they  are  doing.  When  I first  started  using  the 
logs,  there  was  a noticeable  drop  in  the  number 
of  students  who  came  in  every  day  with  a 
“spaced-out"  look  on  their  faces  saying,  “Now, 
what  am  I working  on  in  here?"  The  Student 
Learning  Goals  list  has  not  been  very  helpfuL 
The  Futures  students  don’t  seem  to  like  it,  and 
they  rarely  check  off  anything  on  it.  I believe  it 
could  be  improved  by  cutting  down  the  number 
of  skills  and  topics  and  adding  some  more  light- 
hearted ones.  I also  think  that  many  of  the 
students  honestly  do  not  feel  themselves  to  be 
interested  in  anything  that  one  could  learn 
about  in  school,  and  may  not  until  they  get 
older  or  their  interest  is  somehow  sparked  by 
the  teacher. 
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The  Process  Continues 

The  process  we  began  last  year  will  con- 
tinue for  a long  time.  As  the  months  go  by,  we 
focus  more  on  some  of  the  questions  that  were 
left  out  of  our  initial  discussions:  How  do  we 
manage  the  logistics  of  using  the  forms  and  hold 
individual  meetings?  How  do  we  adapt  learner- 
centered  assessment  ideas  developed  through 
working  with  adults  to  the  young  people  we 
serve?  These  questions  will  no  doubt  lead  us  to 
reassess  and  change  our  approach  in  the  future. 


Note 

1.  Adapted  from  Germanoski,  M.  “The  Education  Goals 
Assessment  Packet,”  Adventures  in  Assessment  Vol. 
I,  p.  33-36. 
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a continuing  feature  on  math  assessment 


The  "Whole-Person"  Approach 

in  Math  Assessment1 


by  Mary  Jane  Schmitt  and  Helen  Jones 


Math  is  not  an  isolated  activity,  some- 
thing which  adults  do  only  when 
confronted  with  taking  a test  or 
demonstrating  a competency.  Mathematics  is  a 
dynamic  process,  one  that  is  rich  and  diverse, 
one  that  involves  gathering  information,  explor- 
ing ideas,  discovering  relationships,  identifying 
patterns,  and  making  connections.  We  believe 
that  by  talking  about  and  teaching  math  in  this 
way,  adult  learners  will  come  to  value  math- 
ematics, be  confident  in  their  ability  to  do  math, 
become  real-life  problem  solvers,  and  learn  how 
to  reason  and  communicate  mathematically.2 

What's  Wrong  with  this  Picture? 

An  adult  learner  walks  tentatively  into  an 
ABE  math  classroom  and  is  greeted  warmly  by 
the  teacher.  The  other  learners  move  over  to 
make  some  room.  The  atmosphere  is  comfort- 
able and  accepting.  After  introductions,  the 
teacher  approaches  the  new  arrival  with  a basic 
math  assessment  (computation  and  word  prob- 
lems), and  asks  the  learner  to  try  some  of  the 
problems.  She  explains  that  her  goal  here  is  to 
find  out  what  the  adult  learner  remembers  and 
where  s/he  needs  to  start  reviewing. 

What’s  wrong  with  this  picture?  Nothing  at 
all,  unless  this  is  all  you  do  to  assess  your  stu- 
dents’ knowledge  and  past  experience  with 
math.  Computational  tests  are  helpful  but  they 


can’t  provide  all  the  information  you  need.  To 
plan  the  most  effective  learning  activities  and 
create  the  right  kind  of  learning  environment, 
consider  taking  a “whole-person”  approach  in 
the  initial  math  assessment. 

The  following  nine  factors  are  important  to 
consider  when  creating  an  individual’s  learning 
plan: 

Factor  1 

Emotions  and  Attitudes 

Factor  2 

Long  and  Short-Term  Goals 

Factor  3 

Everyday  Math  Experiences 

Factor  4 

Computational  and  Problem-Solving 
Processes 

Factor  5 

Math  Skill  Level 

Factor  6 

Reading  Level 

Factor  7 

Cultural  Background 
Factor  8 

Learning  Pace  and  Style 

Factor  9 

Perceptual  Disabilities  or  Strengths 
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Questions  for  Reflection 

• How  openly  do  we  talk  about  math 
and  past  experiences  with  math  in  the 
classroom? 

• What  information  do  I need  to  know 
about  my  students  and  what  are  some 
ways  I can  elicit  that  information? 

• Ultimately,  how  do  I incorporate  this 
information  into  my  instruction  so 
that  I build  upon  the  learners’  knowl- 
edge? 

• Do  I start  each  learner  “where  s/he’s 
at,"  neither  overestimating  nor 
underestimating  his  or  her  abilities? 

• How  do  I know  when  my  students 
are  not  “getting  it?” 


Figure  1 

FACTOR  1:  Emotions  and  Attitudes 

Many  learners  have  had  negative  experi- 
ences with  math;  it’s  important  for  you  to 
recognize  this  and  to  open  up  your  classroom 
for  sharing  “math  memories”— your  students’ 
and  your  own.  Keeping  a math  journal  is  one 
way  learners  can  record  their  thoughts  and 
feelings  about  doing  math,  while  also  describing 
what  they’re  learning  in  class. 

Students  who  are  math  anxious  often  have 
developed  coping  mechanisms  to  deal  with  their 
fears:  panicking  when  faced  with  a word 
problem,  saying,  “I  hate  math,”  or,  “I  can’t  do 
math,”  or  hiding  how  they  get  an  answer 
because  they  fear  it’s  “not  the  right  way.”  It’s 
important  to  acknowledge  this  “emotional” 
information  and  to  deal  with  it  in  your 
classroom. 


Let  students  know  that  you  are  fully  confi- 
dent in  their  ability  to  do  math,  and  encourage 
them  to  insist  upon  “getting  it.”  They  should  be 
comfortable  in  telling  you  when  they’re  con- 
fused. Agree  upon  a signal,  like  holding  up  a 
pencil,  to  let  you  know  if  something  doesn’t 
make  sense.  Ask  questions  along  the  way  to 
check  comprehension.  Remember,  keep  the 
dialogue  going. 

Reflect  on  your  feelings  about  math.  If  you 
are  at  all  uncomfortable  with  math,  think  about 
how  your  own  math  anxiety  plays  out.  What 
kind  of  messages  do  you  give  your  students  Do 
you  go  too  much  by  the  book  because  you're 
afraid  to  make  a mistake?  It’s  best  to  be  honest 
and  to  learn  with  your  students,  (see  Fig.  1) 

FACTOR  2:  Long  and  Short-Term  Goals 

It  is  critical  to  know  why  a learner  has 
come  back  to  school.  Knowing  your  students’ 
goals  will  help  you  develop  and  guide  the  overall 
curriculum.  This  is  information  the  teacher 
keeps  in  mind  as  s/he  begins  instructional 
planning. 

Setting  up  a plan  that  enables  students  to 
achieve  their  long-term  goals  is  a critical  next 
step.  Some  outside  research  on  your  part  may  be 
necessary.  If  a student  wants  to  go  to  nursing 
school,  for  example,  get  a sample  of  the  entrance 
exam  and  learn  about  the  entrance  requirements. 
You  can  find  copies  of  current  nursing  text- 
books and  get  a feel  for  the  content  of  the 
course  work. 

Be  skeptical  of  exam  preparation  books.  Go 
to  the  source,  if  you  can.  If  the  student  wants  to 
take  the  GED,  get  the  Official  Practice  Tests 
and  teacher’s  guide  that  come  directly  from  the 
GED  Testing  Service. 

Don’t  forget  about  short-term  goals.  They 
are  critical  benchmarks  of  progress.  The  learner 
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who  wants  to  be  a nurse,  for  example,  can  work 
toward  the  short-term  goal  of  learning  the 
metric  system.  Also,  students’  goals  may  change 
over  time,  so  be  sure  to  check  in  periodically. 

FACTOR  3:  Everyday  Math  Experiences 

Asking  learners  to  think  of  ways  they’ve 
used  math  in  the  past  month  is  a good  activity 
to  initiate  for  many  reasons.  It  opens  up  discus- 
sion, and  allows  learners  to  see  the  many  ways 
they  use  math  every  day.  A typical  list  might 
include: 

• doubling  a recipe 

• figuring  sales  tax 

• building  a porch 

• measuring  baby  formula 

• grocery  shopping 

• catching  a bus 

• balancing  a checkbook 

• punching  a time  card 

• analyzing  a paycheck 

• checking  change 

• measuring  food  for  intake  (in  the  case  of  a 
diabetic) 

• looking  for  an  apartment 

Keep  track  of  the  information  your  stu- 
dents provide,  and  use  it  in  your  instruction. 
Mathematics  is  about  making  connections,  and 
good  lessons  start  with  experiences  students  can 
relate  to. 

FACTOR  4:  Computational  and 
Problem-Solving  Processes 

Observing  and  listening  to  how  students 
solve  problems,  and  affirming  their  strategies  is 
important.  Students  should  know  early  on  that 
there  are  many  ways  to  reach  a solution,  and 


that  the  best  way  is  that  which  makes  most 
sense  to  them. 

Acceptance  of  learners’  calculating  methods 
does  not  mean  that  you  should  accept  all  an- 
swers or  off-the-wall  thinking.  First,  find  out 
how  the  learner  arrived  at  that  answer  before 
you  challenge  or  correct  him  or  her.  Try  to 
understand  the  logic  behind  each  learner’s 
approach.  The  more  you  listen  to  and  respect 
the  particular  way  of  reasoning,  the  easier  the 
teaching  and  learning  process  will  be.  (see  Fig.  2) 

FACTOR  5:  Math  Skill  Level 

It’s  important  to  know  if  your  students 
know  the  four  operations  on  whole  numbers, 
fractions,  and  decimals,  and  whether  they  can 
use  these  skills  in  context.  That’s  why  we 
usually  start  out  with  a computational  math 
assessment. 

Be  skeptical  of  this  assessment  process, 
though;  it  is  not  an  exact  science.  Even  in  your 
best  attempts,  you  may  underestimate  or 
overestimate  a learner’s  skill  level  or  potential. 

A student  who  can’t  divide,  for  example,  may  be 
able  to  solve  algebra  problems. 

FACTOR  6:  Reading  Level 

Math  will  never  make  any  sense  if  language 
gets  in  the  way.  It’s  important  that  word  prob- 
lems be  within  the  learners’  vocabulary  and 
comprehension  level.  Also,  when  integrating 
reading,  writing,  and  math  in  your  curriculum, 
be  sure  to  consider  the  learners’  abilities  in  all 
three  areas.  However,  don’t  let  low  reading 
levels  prevent  you  from  teaching  math.  If  you 
are  working  with  new  readers,  present  problems 
orally  in  groups,  or  tape-record  problems  for 
new  readers  to  work  with.  There  are  ways  to 
teach  math  successfully  to  learners  who  are  at 
varying  reading  levels. 
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Here  are  some  ways  to  make 
the  Initial  math  assessment 
more  accessible  to  the  learner. 

ALLOW  the  learner  to  use  a variety  of 
computational  processes:  paper  and 
pencil,  mental  math,  manipulatives 
(concrete  objects),  or  a calculator. 


LIMIT  the  number  of  items  on  the 
initial  assessment  to  25,  and  the 
amount  of  time  spent  with  the  test  to 
one  half  hour.  Adults  returning  to 
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comprehensive  test  that  shows  them 
how  much  they  don’t  know  or  don’t 
remember. 


WRITE  each  problem  on  a “3  x 5” 
card.  Ask  the  learner  to  separate  the 
cards  into  three  piles:  (a)  I definitely 
know  how  to  do  these;  (b)  I’m  not 
sure;  (c)  I definitely  don’t  know  how 
to  do  these. 


ASK  the  student  to  write  “twenty- 
one”  as  many  ways  as  s/he  can.  This 
will  give  you  some  idea  of  his  or  her 
sophistication  with  numbers.  For 
example,  “7  x 3”  or  “20  + 1”  is  not  as 
sophisticated  as  “19.9  + 1.1”  or 

“ \46o~  + Y T"  ”. 

ASK  students  to  write  word  problems 
using  the  computation  problems  on 
the  test. 


TALK  about  the  test  and  the  strategies 
used  to  solve  problems.  Remember, 
communication  is  an  integral  part  of 
doing  math. 


FACTOR  7:  Cultural  Background 


U.S.  METHOD  OF 

Some  other 

DIVIDING  DECIMALS: 

countries: 

4.2  ) 14.68 

14,68  ) 14,2 

Figure  3 


Many  approaches  to  doing  basic  math  are 
fairly  universal.  However,  some  major  cultural 
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learners  who  attended  schools  in  a Spanish-, 
French-,  or  Portuguese-speaking  country  may 
write  “two  thousand”  as  “2.000,”  and  write; 
“two”  as  “2,000.”  Also,  their  method  for  sub- 
traction may  not  involve  the  concept  of  “bor- 
rowing.” In  some  countries,  division  is  also 
done  differently.  Measurement  systems  also  vary 
by  country.  Many  countries  use  the  metric 
system,  so  you  may  need  to  use  12-inch  rulers 
and  yardsticks  to  show  the  customary  units  of 
measure  in  the  United  States.  When  working 
with  students  try  to  learn  the  methods  they 
were  taught.  Have  students  from  the  same 
country  tutor  each  other.  In  the  classroom, 
build  on  the  students’  methods  if  you  can. 

Don’t  insist  on  their  learning  new  algorithms  if 
their  own  methods  work  just  as  well. 

If  the  students  have  children  in  American 
schools,  talk  about  how  math  is  taught  in  the 
United  States.  You  can  create  interesting  multi- 
cultural math  lessons  using  this  information. 

FACTOR  8:  Learning  Pace  and  Style 

People  learn  at  different  rates.  That’s  a fact 
of  life!  So,  many  adult  learning  centers  provide 
individualized  instruction  to  ensure  that  learners 
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have  enough  time  to  absorb  the  material. 

Watch  out  for  this.  Instruction  that  is 
completely  individualized  can  be  isolating.  By 
pairing  students  who  learn  at  similar  paces,  or 
by  forming  small  work  groups,  you  can  provide 
for  an  enriching  and  cooperative  learning  experi- 
ence. 

Adults  also  have  many  different  learning 
styles.  In  the  ABE  classroom,  some  learners  are 
self-directed,  whereas  others  need  continual 
support  from  the  teacher.  Some  learners  prefer 
to  work  alone,  just  as  others  learn  best  in  small 
groups.  Observe  your  students,  and  learn  how 
they  work  best. 

Consider  your  own  teaching  style.  Are  you 
directive  or  facilitative?  Does  your  teaching  style 
match  the  learning  styles  of  most  of  your  stu- 
dents? These  are  some  issues  you  ought  to  take 
into  account. 

FACTOR  9:  Perceptual  Disabilities  or 
Strengths 

Many  teachers  are  familiar  with  the  follow- 
ing patterns:  A learner  has  difficulty  memoriz- 
ing the  times  tables,  is  unable  to  pay  attention 
to  “ + , -,  x,  4-  ” symbols,  or  can’t  line  up  deci- 
mal points.  The  list  goes  on.  These  may  be 
symptoms  of  a visual  perception  disability. 

If  a learner  has  problems  following  oral 
directions,  or  has  difficulty  counting  out  loud, 
s/he  may  have  an  auditory  deficit.  Although 
they  do  not  account  for  all  problems,  learning 
disabilities  certainly  do  exist.  It  is  well  worth 
your  time  to  learn  more  about  this  important 
issue. 

Just  as  you  should  be  aware  of  possible 
disabilities,  it  is  equally  important  to  build  on 
students  perceptual  strengths.  In  your  instruc- 
tion, include  the  kinesthetic,  visual,  and  audi- 
tory learning  modalities. 


Summary 

Good  teaching  begins  with  good  assessment 
and  improves  with  appropriate  and  continuous 
evaluation.  A whole-person  approach  to  math 
assessment  takes  into  account  not  only  the 
learners’  “pencil  and  paper”  skills,  but  also  their 
feelings  about  math  and  everyday  approaches  to 
problem-solving. 


Notes 

1.  This  article  is  excerpted  from  the  viewer’s  guide  which 

accompanies  the  training  video,  “Changing  the 
Rules:  Teaching  Math  to  Adult  Learners,”  available 
from  New  Readers  Press. 

The  video  demonstrates  four  key  ideas  to  improve 
adult  basic  mathematics  instruction.  These  include 
using  a “whole-person”  approach  to  student  assess- 
ment;  integrating  concrete  learning  activities  into 
classroom  instruction;  using  real-life  math  problems 
that  have  relevance  to  adults’  daily  lives;  and  finally, 
using  a spiral  approach  to  teach  math  content. 

2.  These  goals  are  also  outlined  by  the  "National  Council 

of  Teachers  of  Mathematics  in  Curriculum  and 
Evaluation  Standards  for  Mathematics”,  NCTM 
(March,  1989). 
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Creating  Change  or  Creating 
Accessibility:  A Dialogue 


with  Lindy  Whiton  and  Loren  McGrail 


In  January,  1992,  the  Bureau  of  Adult  Educa- 
tion (BAE)  of  the  Massachusetts  Depart- 
ment of  Education  initiated  the  GOALS 
Project,  a field  test  of  10  varied  components 
designed  to  lead  to  the  development  of  an 
Accountability  System  for  adult  basic  education 
programs  in  Massachusetts.  The  purpose  of  the 
GOALS  Project  according  to  Sandy  Brawders, 
Supervisor  of  Development,  BAE,  is  to  “define  a 
set  of  indicators  of  program  quality  which  are 
educationally  sound,  useful  to  students,  teachers 
and  funders,  and  not  excessively  time- 
consuming.” 

Component  3 of  the  GOALS  Project  is 
working  with  1 1 programs  using  alternative 
forms  of  assessment.  Lindy  Whiton,  Coordina- 
tor of  Component  3,  describes  the  theoretical 
and  philosophical  basis  of  Component  3 as 
follows: 

“We  all  believe  in  learner’s  goals  being  not 
only  important,  but  a driving  force  in  our 
curriculum;  thus,  our  assessment  practices  are 
based  on  the  individual  goals  of  the  learners  as 
well  as  the  goals  of  the  individual  programs. 
Whether  we  are  teaching  ABE,  GED,  or  ESL, 


we  believe  that  literacy  practices  have  to  directly 
relate  to  not  only  the  goals  of  learners,  but  their 
immediate  lives.”  (Whiton,  Component  3,  First 
Draft,  p.  1)  Based  on  these  beliefs,  each  of  the  11 
programs  in  Component  3 “have  developed 
assessment/evaluation  tools  that  take  into 
consideration  who  their  learners  are,  who  their 
teachers  are,  the  goals  of  their  learners,  their 
interests,  and  how  they  use  literacy  in  their 
daily  lives.”  {Ibid.) 

The  following  conversation  with  Lindy 
Whiton  was  conducted  at  the  beginning  of  the 
summer  of  1992  with  Loren  McGrail,  editor  of 
Adventures  in  Assessment.  Minimal  changes  have 
been  nude  to  the  original  transcript  so  that  the 
flavor  of  the  dialogue  could  be  retained. 

Component  1 of  the  GOALS  Project  uses  stan- 
dardized tools  to  measure  student  progress. 
Before  we  begin  talking  about  Component  3, 
could  you  please  talk  about  Component  If 

Component  1 is  set  up  so  that  you  collect 
profile  information  on  an  intake  of  a student, 
everything  from  race  to  past  educational  experi- 
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ence  to  work  history.  What  you  do  in  the  very 
beginning  is  to  give  the  CASAS*  pre-test  and 
you  get  a score  from  the  CASAS  test  and  then 
people  go  about  their  normal  lives  and  their 
programs  track  only  participation  and  atten- 
dance. Then-I  think  it's  every  six  weeks,  but 
I'm  not  positive-they  do  another  CASAS  to  test 
whether  there  has  been  growth.  Now  that's  one 
track.  The  idea  is  to  see  whether  competency- 
based  tests  will  determine  whether  or  not  a 
learner  is  meeting  their  goals  [as  defined  by 
competencies].  This  is  only  being  tested  for  ABE 
learners  right  now.  Then  they  will  compare  this 
information  with  student  participation  by 
looking  at  attendance  scores.  They'll  be  able  to 
look  at  CASAS  quantitative  markings  and  see 
whether  or  not  learners  are  reaching  their  goals, 
thus  [find  out  if]  programs  are  fulfilling  their 
duties/ responsibilities. 

Compared  to  Component  1,  what  is  the  differ- 
ence in  design  or  intent  of  Component  3 f 

Well,  there  was  no  way  you  could  measure 
what  happens  in  a Component  3 program  with 
the  tools  that  they  were  using  to  measure  Com- 
ponent 1.  What  we  decided  was  that  the 
method  in  which  we  did  Component  3 should 
reflect  the  theoretical  foundations  of  those 
[participating]  programs.  If  I was  going  to  run  a 
component  that  used  alternative  assessment  and 
participatory  approaches  or  whole  language 
approaches,  then  I’d  better  run  that  component 


in  the  same  methodological  way. 

I’ve  completely  allowed  the  programs  to 
determine  what  we’re  doing  and  what  we’re 
collecting.  When  we  first  got  together  I led 
them  through  a couple  of  activities  just  to  get 
them  to  talk  about  assessment  and  evaluation 
and  what  the  difference  is  between  assessment 
and  evaluation.  What  do  you  mean  when  you 
talk  about  evaluation?  What  do  you  mean 
when  you  talk  about  assessment?  There  were  a 
lot  of  us  in  the  room  that  day,  not  only  pro- 
gram people,  so  there  were  a lot  of  different 
perspectives.  It  was  exciting.  People  fought.  I 
mean  there  was  iust  a sort  of  fiVhring  KarW  and 
forth  and  really  getting  into  it.  The  idea  was  to 
create  a kind  of  community  quickly  and  trust  in 
that  fighting,  that  it  was  okay.  What  it  did  was 
lay  out  where  people  were  coming  from,  their 
theoretical  foundations  came  out  immediately. 
You  can’t  evaluate,  you  can’t  use  a test  that  is 
totally  skills-based  to  measure  a curriculum  that 
is  not  skills-based.  Those  kind  of,  issues  came 
out  immediately,  so  that  we  all  knew  where  we 
stood  in  that  realm.  Then  they  decided  as  a 
group  the  thing  to  do  next  was  to  meet  again 
and  share  with  each  other  what  each  program 
did  on  intake,  ongoing,  and  final  evaluation  of 
learners,  teachers,  classrooms,  and  programs. 
They  shared  them  all  and  then  they  sent  me 
away  with  the  task  of  putting  it  together  in 
some  sort  of  order  and  writing  up  a paper  that 
explains  what  their  foundation  is,  what  those 


* The  Comprehensive  Adult  Student  Assessment  System  is  “a  comprehensive  educational  assessment  system 
designed  to  measure  competency-based  curriculum  for  all  levels  of  Adult  Basic  Education  and  English  as  a 
Second  Language,  including  a pre-vocational  curriculum.”  (CASAS  Overview,  p.l)  CASAS  “measures  func- 
tional basic  skills  in  Reading,  Math,  and  Listening  Comprehension,  utilizing  an  Item  Bank  of  more  than  5,000 
multiple-choice  test  items.  CASAS  also  includes  authentic  or  applied  performance  measures  on  oral  profi- 
ciency, writing  skills,  pre-employment  and  work  maturity  competence,  and  critical  thinking  skills.”  (CASAS, 
Training  Workshop  Handouts,  1990,  p.  8d) 
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three  pieces  are.  I used  Elsa  [Auerbach’s]  and 
your  stuff  [framework  for  Adventures  in  Assess- 
ment: getting  started,  ongoing,  and  looking 
back]  to  organize  the  notebook. 

You  said  there  was  kind  of  a discussion  about 
differences  in  assessment  and  evaluation.  Did 
you  come  to  consensus  about  these  differences? 

No.  And  we  allowed  it  not  to  come  to 
consensus.  We  were  all  sort  of  split  and  in  the 
end  made  the  decision  that  we  all  agreed  upon 
what  the  concepts  were,  but  not  the  labels  for 
those  concepts. 

And  what  were  those  concepts? 

We  looked  at  assessment  as  something  you 
did  [on  an]  ongoing  [basis]-something  you  did 
daily  like  a learning  log  or  a teaching  log  that 
you  daily  put  your  perceptions  on  what  hap- 
pened. That  was  assessment.  Evaluation  was  at 
the  end  of  the  cycle  looking  at  doing  miscue 
analysis,  doing  a program  evaluation  of  the 
whole  class.  What  were  your  strengths?  Looking 
back  at  larger  things... Some  people  switch  the 
two  concepts  around  so  that  they  were  calling 
evaluation  what  you  and  I call  assessment. 

So  you  decided  not  to  quibble  over  the  terms , 
but  just  made  sure  those  concepts  were... 

...agreed  upon. 

Wasn’t  it  true  that  you  decided  not  to  have 
people  other  than  the  participants  in  your 
component  come  to  the  meetings? 

The  group  made  the  decision  to  ask  that 
other  people  not  come  and  that  we  allow  our- 
selves the  privacy  to  discuss  what  does 
happen... Issues  needed  to  be  talked  about  within 
the  group  without  anybody  overseeing... Like 
some  of  these  tools  have  been  developed  but  not 


put  into  practice  unfortunately.  Programs 
struggle  with  how  to  fit  this  into  part-time 
teacher  status.  I think  some  of  the  programs  feel 
that  they  have  very  strong  intake  procedures- 
and  they  do— and  that  their  curriculum  is  very 
good,  and  they  sort  of  have  a hard  time  teaching 
math,  and  they  don’t  have  any  means  of  evaluat- 
ing anything  they  do-so  what  they’d  like  to  be 
able  to  do  is  say  this  is  the  help  we  need.  This  is 
where  we  think  we  are.weak.  We’re  definitely 
strong  here.  These  are  not  reasons  to  not  get 
refunded. 

So  your  belief  is  that  they  are  worried  about 
giving  out  information  and  letting... 

I think  they  are  worried  about  it  now  and 
the  idea  is  to  find  a way  to  open  up  that  dia- 
logue. The  answer  to  that  question  will  then 
become:  “This  is  who  you  should  call  at 
SABES,”  or  “Here  is  this  resource  to  help  you 
develop/strengthen  this  piece.” 

In  Component  1,  learner  goals  are  matched 
with  competencies  from  the  CASAS  competency 
list.  How  are  these  goals  identified  in  Compo- 
nent 3? 

The  majority-this  is  where  the  uniqueness 
of  the  programs  come  into  play-ask  the  learners 
what  their  goals  are.  They  may  start  with  them 
at  the  beginning  of  a cycle  or  they  may  plug 
them  into  a certain  part  later  in  the  cycle  so  the 
curriculum  is  determined  by  those  goals. 

So  then  there’s  more  of  a direct  relationship 
between  the  curriculum  and  those  goals? 

The  Log  School  is  probably  the  most  direct, 
[see  p.  11]  When  they  get  a group  of  people  in, 
they  take  those  goals... that  are  most  in  common 
across  all  learners  [and  that]  is  where  they  start 
their  curriculum.  So  that's  the  theme  or  the 
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focus  that  they  start  in  and  then  it  goes  on  from 
there.  The  participants  direct  where  the  curricu- 
lum goes.  In  the  end  the  students  do  the  evalua- 
tion: Did  your  goals  get  met?  Where  do  you 
think  the  strengths  of  the  program  lay?  Where 
do  you  think  it  could  be  stronger?  They  ask 
them  directly  instead  of  trying  to  determine 
some  competency  that  they  standardized. 


How  does  a curriculum  that  is  already  set  up  fit 
in  with  an  approach  that  tries  to  find  out  what 
the  learner's  goals  are?  Wouldn't  that  be  the 
same  problem  or  configuration  as  Component  1 
where  there  is  something  pre-designed  to  meet 


something  that  is  coming  from  the  learners? 

In  Component  3 there  are  three  programs 
or  four  where  that  is  true.  I think  that  the 
Quincy  School  would  say  that  people’s  goals  are 
to  learn  English  and  that  they  have  a standard 
way  of  doing  that.  One  of  the  reasons  that  they 
chose  to  participate  in  this  particular  component 
is  that  they’re  interested  in  changing  their 
assessment  policies. 


And  changing  that  attitude? 

Yeah.  There  seem  to  be  reasons  to  keep  it 
and  reasons  to  get  rid  of  it  and  they,  as  a staff, 
are  having  that  debate.  So  they  asked  to  be  pan 
of  that  larger  debate. 


Right.  And  what’s  been  real  interesting  to 
me  is  the  dialogue  that’s  been  going  on  between 
the  programs... You  might  be  able  to  use  Com- 
ponent 1 on  the  Quincy  School  as  it  stands  now 
but  not  on  Read/Write  Now. 

Are  you  saying  that  because  there  was  no 
operative  definition  for  alternative  assessment 
to  start  with,  people  chose  this  component 
because  they  felt  they  were  doing  alternative 
assessment  and  because  they  wanted  to  make  a 
change? 

Yes. 

...And  that  the  Department  of  Education  made 
their  selection  based  on  wanting  some  of  those 
programs  who  wanted  to  change  to  be  in  the 
company  of  others  who  have  already  made  the 
change? 

Yes.  And  what  I think  may  be  possible-this 
isn’t  in  writing  yet-but  I think  that  what  we’U 
do  is  once  we  [complete  this  research],  we’ll  get 
ten  other  programs  that  are  in  the  same  spot 
who  want  that  change  but  need  help  developing 
their  tools  and  adapting  them  to  their  own 
programs. 

Are  you  saying  you  all  would  act  as  mentors? 

Yes. 


Then  not  everyone  in  Component  3 is  already 
doing  alternative  assessment?  There  are  people 
who  are  coming  into  it  because  they  want  to? 

Well,  Quincy  School  defines  themselves  as 
doing  alternative  assessment  because  they  made 
up  their  own  tools.  They  don’t  use  anything 
published. 

So  there  are  two  definitions  of  alternative 
assessment? 


One  of  your  discoveries  is  that  program-based 
assessment  does  not  necessarily  mean  alterna- 
tive assessment  if  it's  following  the  same 
theoretical  principles  as  those  of  standardized 
tests  which  are  primarily  skills-based.  I think 
this  is  the  middle  ground  where  most  of  the 
state  is.  It's  almost  as  if  there  is  a missing 
component  in  the  GOALS  project-program 
based  assessment-and  its  showing  up  in 
Component  3.  Program-based  assessment  does 
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not  necessarily  mean  alternative  assessment.  It 
means  alternative  to  standardized  testing.  So 
maybe  we  should  stop  using  the  word  “alterna- 
tive” and  use  something  like  “authentic”  or 
“learner  centered”  or  “participatory”  to  make 
the  distinction. 

Yeah.  I think  that’s  a good  point.  I think  it 
is  true  that  when  we  chose  programs  to  go  into 
Component  3,  we  chose  a few  of  one  kind  and  a 
majority  of  learner-centered. 

Do  you  see  a pattern  among  people  who  have 
either  been  through  some  university  training  or 
influenced  by  universities  in  the  development 
of  the  kind  of  tools  that  are  developed? 

I think  the  majority  of  the  people  in  Com- 
ponent 3 have  been  influenced  by  either  Boston 
University  or  the  University  of  Massachusetts. 

I think  that’s  part  of  the  difference  between 
Components  1 and  3. 

Are  there  other  themes  or  issues  that  have 
emerged? 

There  are  two  fears.. .the  group  [has].  One 
is  if  you  develop  these  things-tools-are  they  just 
going  to  become  standardized?  [Another  is:] 
How  do  you  teach  pan -time  teachers  how  to  use 
these  types  of  tools  which  will  [eventually]  fit 
their  particular  populations?  Are  we  just  going 
to  find  Janet  Kelly’s  stuff  all  over  the  place 
without  any  adjustment  made  to  who  we  are 
serving? 

...  Who  they’re  serving  and  what  type  of  literacy 
they  are  practicing?  It  seems  to  me  one  of  the 
other  fears,  which  may  or  may  not  be  yours,  is 


that  people  will  take  tools  and  impose  them 
upon  teaching  situations  that  are  not... 

...meant  to  be  measured. 

In  fact,  they  would  be  better  measured  with 
more  standardized  tools  because  of  the  ways  in 
which  they  are  teaching. 

Right,  I think  that’s  true. 

Maybe  that’s  not  a concern  of  the  group’s  but 
maybe  more  of  yours? 

Well,  I think  the  group  is  looking  at  how 
to.. .literally,  physically  create  that  toolkit  and 
create  little  abstracts  on  why  this  tool  works 
best  in  this  type  of  situation.  You  won’t  have  an 
SRA*  box  full  of  tools,  but  rather  a way  that 
helps  people  say,  “I  want  to  know  whether  my 
students  are  getting  the  connections  between 
what  they’re  writing  and  what  they’re  read- 
ing...” 

It  seems  to  me  one  of  the  real  benefits  that 
Component  3 could  give  all  of  the  other  compo- 
nents is  a deeper  and  richer  understanding  of 
what  goals  are,  who  the  learner  is  in  relation- 
ship to  their  own  goals  and  some  really  appro- 
priate ways  in  which  that  gets  measured  for  the 
learner’s  sake.  Do  you  see  that  as  part  of  your 
agenda  or  is  that  an  agenda  of  anyone  in  your 
group?  From  my  viewpoint,  that’s  the  piece  of 
research  which  I think  could  be  very  helpful  to 
the  Bureau. 

What  I’m  trying  to  do  is  create  a system 
that  allows  people  who  think  like  Component  1 
to  understand  the  benefit  of  Component  3.  If 
you  change  the  language  and  the  whole  way  in 


* Science  Research  Associates  (1968-1986)  - graded  reading  passages  with  multiple  choice  comprehension 
questions. 
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which  you  talk  about  it  entirely  and  leave 
nothing  for  comparison,  then  you  leave  no 
access  into  your  ideas.  If  what  I can  do  is  create 
this  line  of  data  that  is  comparable  in  the  same 
way  [as  the  other  components]  then  it  gives  the 
Bureau-and  everybody-access  to  what  you  and  I 
think  are  the  important  issues.  I’ve  got  to  create 
comparable  data  in  a research/quantitative  way. 

Even  though  you  have  a totally  different  theo- 
retical basis,  you  still  think  you  can  compare 
data? 

I can  create  access  into  the  more  important 
meat.  Once  you  can  get  a hold  of  this  under- 
standing then  things  like  understanding  that  we 
are  not  tracking  learners... 

...but  following  their  tracks... 

...becomes  more  easily  understood  by 
somebody  who  thinks  like  this.  Otherwise  we’ll 
get  thrown  out  as  hippies. 

Well,  it’s  the  classic  argument.  Who  needs  to 
change  what  according  to  whom?  I guess  I’m 
suggesting  that  your  criteria  and  values  and 
your  ways  of  looking  at  goals  should  not 
change. 

I’m  not  disagreeing  with  you. 

Well  you  kind  of  are. 

I’m  saying  that  we  need  to  go  slower-that 
we  can’t  change  them  until  they  understand  us. 
Before  [we]  create  change,  we  have  to  create 
accessibility. 
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Letter 


Keeping  Assessment  Out  of  Program 
Accountability 

The  search  for  alternative  methods  for 
measuring  the  progress  of  adult  learners  started 
with  dissatisfaction  with  standardized  tests  in 
measuring  the  many  different  and  important 
ways  in  which  adults  learn.  The  alternative 
approaches  to  measuring  student  progress  that 
have  been  explored  in  Adventures  in  Assessment 
may  soon  be  recognized  as  acceptable  alterna- 
tives to  standardized  tests,  and  this  offers  the  ' 
hope  that  assessment  will  become  a more  valu- 
able experience  for  learners  and  teachers.  Unfor- 
tunately, these  new  tools  may  become  part  of 
the  program  accountability  systems  used  by 
funding  agencies  to  judge  whether  or  not  their 
money  is  being  well  spent. 

Last  year  Sondra  Stein  and  I published  an 
article  in  Adult  Learning  (“How  a College  and 
University  Model  Could  Be  Used  to  Judge 
Program  Effectiveness, " Sept.  1991,  Vol.  3, 

No.  1)  that  suggested  an  accreditation  approach 
to  measuring  program  effectiveness  for  account- 
ability purposes.  Our  intent  was  less  to  promote 
the  accreditation  model  than  to  argue  for  drop- 
ping student  assessment  from  the  program 
accountability  process.  Using  student  progress 
as  a measure  of  program  effectiveness  can  cause 
programs  to  seek  out  those  students  who  can  do 
well  on  a curriculum  that  teaches  to  a standard- 
ized test  or  to  formulate  curriculum  to  prepare 
students  to  pass  the  test. 


An  accountability  system  must  compare  a 
program  against  a standard  measure  of  impact  in 
order  to  assess  whether  it  is  doing  well  or 
poorly.  If  alternative  assessment  approaches  are 
integrated  into  an  accountability  system,  these 
tools  will  eventually  be  required  to  perform  like 
standardized  tests  and  provide  measures  that  can 
judge  a program  against  some  standard.  Only 
standardized,  comparable  measures  can  serve  as  a 
measure  of  program  effectiveness. 

A program  should  be  concerned  with 
improving  its  standard  of  service  rather  than 
recruiting  students  who  will  be  successful  or 
training  students  to  be  successful  on  a standard- 
ized test.  Student  assessment  should  take  place, 
but  it  should  be  designed  to  produce  measures  of 
progress  that  serve  the  needs  of  students  and 
teachers,  not  the  needs  of  a program  to  secure 
funding.  Using  student  assessment  as  the  mea- 
sure of  effectiveness  for  program  accountability, 
no  matter  how  good  the  assessment  tool,  will 
always  make  the  test  result  the  focus  of  pro- 
grams rather  than  the  needs  of  the  student. 

Funding  agencies  do  have  a legitimate  right 
to  measure  the  effectiveness  of  the  programs 
they  fund.  But,  looking  at  student  progress  does 
not  necessarily  provide  a way  to  judge  whether 
or  not  money  is  being  well  spent.  Some  of  the 
existing  standardized  assessment  tools  are  good 
measures  of  student  progress  in  acquisition  of 
skills  or  competencies,  but  the  impact  on  a 
student’s  life  could  be  in  areas  not  measured  by 
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the  assessment  tool.  Though  progress  as  mea- 
sured by  tests  might  come  slowly  for  some 
students,  positive  changes  in  their  lives  could  be 
occurring. 

Judging  a program  against  standards  of 
practice  and  service  can  give  an  indication  of 
how  well  money  is  being  spent.  If  a program  has 
all  of  the  elements  of  good  practice  and  service, 
then  students  who  enter  and  remain  in  the 
program  should  be  doing  about  as  well  as  they 
can.  Just  as  higher  education  programs  are 
judged  on  the  quality  of  their  teachers,  curricu- 
lum, materials,  administration,  and  equipment- 
adult  basic  education  should  be  measured  by  the 
elements  of  its  programs.  The  accreditation 
model  used  by  colleges  and  universities  does  not 
rely  on  student  assessment.  Why  should  ABE 
programs  be  judged  by  student  performance? 

Assessment  of  student  progress  is  now 
serving  as  a “red  herring,”  drawing  attention 
away  from  the  need  for  staff  development  and 
program  development.  Time  and  money  is  being 
put  into  the  development  and  implementation 
of  models  for  measuring  student  progress,  many 
of  them  useful  and  effective,  while  little  atten- 
tion is  paid  to  evaluating  and  improving  the 
quality  of  service.  Assessments  of  student 
progress  are  essential  to  managing  learning,  but 
they  should  not  drive  the  system.  Rather,  they 
should  be  an  integral  part  of  a good  system.  The 
need  for  ABE  services  will  be  with  us  for  de- 
cades, and  we  need  a model  for  measuring 
accountability  that  focuses  attention  on  building 
effective  institutions. 

ABE  practitioners  who  are  interested  in 
promoting  alternative  assessment  must  pay 
attention  to  the  context  in  which  assessment 
takes  place.  If  the  context  is  part  of  the  funding 
process,  then  standardized  measures  will  be 
needed.  A better  option  would  be  to  change  and 


improve  the  system  of  measuring  program 
effectiveness  so  that  measures  of  student 
progress  were  not  pan  of  the  funding  process. 
The  assessment  tools,  then,  could  serve  the 
needs  of  students  and  teachers,  and  the  program 
accountability  system  could  focus  on  improving 
service. 

John  P.  Comings 
Vice  President 
World  Education,  Inc. 

210  Lincoln  Street 
Boston,  MA 
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Corrections 


Volume  2,  p.  1 

The  title  on  David  J.  Rosen’s  article  should 
have  read  “Presentation  Portfolio.” 


Volume  2,  p.  74 

Appendix  4 was  incorrectly  identified.  See 
reverse  for  correct  form. 
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Name 


Oats 


Yesterday,  we  wrote  about  Carol.  Carol  was  in  Room  6 when  Janet  was  away. 
You.  and  Yer  wrote. on  the  blackboard. 

When  Carol  came  to  Room  6,  we  learned  English. 

We  wrote  stories  about  ourselves.  We  studied  past,  future  and  present  tens 

We  were  talking  about  what  we  did,  what  we  are  doing  ar.d  what  we  will  do. 

We  made  groups  and  talked  about  pictures  and  we  wrote  the  stories  about  the 
pictures. 


What  do  you  like  to  do  in  Room  6? 


1.  write  about 

pictures 

not  very 

much 

some 

OK 

a 

lot 

2.  write  in  journals 

not  very 

much 

some 

OK 

a 

lot 

3.  write  at  the 

blackboard  together  not 

a lot 

some 

OK 

a 

lot 

4.  read  stories 

that  other 

students  write 

not  very 

much 

some 

OK 

a 

lot 

5.  read  stories 

from  books 

or  magazines 

not  very 

much 

some 

OK 

a 

lot 

6.  speak  English 

together 

not  very 

much 

some 

OK 

Qi 

lotT- 

Please  write  other  ideas  yon  have  about  our  class: 
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Appendix  1 


QUESTIONS  FOR  PROGRAM  EVALUATION 


I.  Are  the  classes  meeting  your  needs? 


2.  Are  the  levels  good? 


3.  Are  there  other  classes  you  would  like  in  the  program?  What  classes? 


4.  Are  the  class  times  good  for  you? 


Do  you  know  people  who  would  like  classes  in  the  morning? 


Do  you  know  people  who  would  like  classes  at  night? 


5.  Sometimes  students  stop  coming  to  classes?  Why? 


6.  What  can  everyone  (students  and  teachers)  do  to  help  students  continue  classes? 


7.  In  addition  to  classes  we  have  child  care,  field  trips,  and  some  special  projects. 
What  other  things  would  you  like  to  see  in  the  program? 
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Appendix  2 


Name 


Community  Learning  Center 
Staff  Program  Evaluation 
May  1992 


49 


INITIAL  ASSESSMENT 

Minimizes  anxiety 

Conducts  assessment  in  learner's 
native  language 

Identifies  learner's  strengths 

Identifies  skills  that  need 
improvement 

Leads  to  development  of  individual 
educational  plan 

Provides  sound  information  for 
placement 

Provides  information  for  support 
services  or  referral,  as 
appropriate 

Discusses  assessment  results  with 
learners  immediately 

Provides  assessment  results  to 
teachers 

Comments ; 


Strength  Need 


?SL 

ABE 

ESL 

ABE 

0 
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CURRICULUM 

Has  curriculum  development  model 
appropriate  to  learner  strengths 
and  needs 


Involves  teachers  in  curriculum 
development 

Is  coordinated  from  level  to  level 


Includes  student  input  in  curriculum 
development 

Comments ; 
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NA 

ESLi ABE 


SUPPORT  SERVICES 


Strength 
ESL  ABE 


Need 
ESL  ABE 


Na 

ESL  ABE 


Offers  enough  counseling  time  for 
students 

Offers  appropriate  personal  counseling 
and  referrals 

Offers  appropriate  education  counseling 
and  referrals 

Offers  appropriate  vocational  counseling 
and  referrals 

Provides  on-site  educational  and 
personal  counseling  in  the  learner's 
native  language,  whenever  possible 
and  appropriate 

Clarifies  goals  and  reviews  need  for 
support  services  on  an  ongoing  basis, 
including  counseling,  childcare  and 
transportation 

Comments ; 


FOLLOW-UP 

Contacts  learners  after  they  complete/ 
exit  program  to  determine  reason  for 
withdrawal  and/or  to  track  learner 
progress  and  achievements 

Supports  learners  in  next  steps 

Comments : 


STAFFING 


Has  well-qualified  staff 

Has  clear  job  description  which 
accurately  reflect  work  done 


Strength 
ESL  i ABE 


ESL 


Need 


ABE 


NA 

ESL, ABE 


Has  appropriate  salaries,  benefits, 
and  opportunities  for  growth 

Demonstrates  good  working  relationships 
among  staff 

Has  diverse  staff  which  reflects 
community 

Has  appropriate  teacher/ learner  ratios 


Has  appropriate  counselor/ learner  ration 


Provides  adequate  staff  supervision  and 
evaluation 

Has  clear  personnel  policies  and  process 
Has  an  Affirmative  Action  policy 
Comments : 


es 


8TAFF  DEVELOPMENT 

Ensures  opportunities  for  all  staff  to 
attend  appropriate  training  and  staff 
development  activities 

Has  individual  and  group  staff 
development  plans 

Has  staff  orientation  process 

COTMnents ; 


&8 


INSTRUCTION 


Is  based  on  learner  goals,  interests, 
and  needs 

Uses  diverse  materials  and  methods 

Uses  materials  and  methods  which  are 
age  appropriate, nongender  biased  and 
interesting 

Includes  some  teacher-made  materials 

Includes  some  learner-  generated 
materials 

Addresses  different  learning  styles 

Assigns  appropriate  homework 

Offers  appropriate  intensity  and 
duration  of  services 

Is  available  at  times  and  locations 
accessible  to  learners 

Lessons  follow  and  build  on  each 
other  in  a way  that  both  teacher 
and  learners  understand 

Comments ; 


ON-GOING  ASSESSMENT 
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Teaches  students  effectively  so  that 
they  progress  at  an  appropriate  pace 

Measures  and  documents  learner  progress 
toward  their  instructional  goals 

Views  goal  setting  as  a continuous 
process 

Produces  clear  indicators  for  level 
changes 

Is  a joint  process  between  learner 
and  teacher 

Does  assessments  regularly 

Uses  appropriate  assessment  tools 

Uses  assessments  to  modify  and 
adapt  curriculum  and  instruction 

Includes  ongoing  diagnostic  assessment 

Comments : 
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PROGRAM  DEVELOPMENT 

Has  a program  development  plan 


Need 


ESL 


ABE 


NA 

ESL 


Includes  student,  staff  and  community 
member  in  planning 


comments ; 


PROGRAM  MANAGEMENT/GOVERNANCE 

Has  regular  staff  meetings 

Has  clear  and  effective  decision-making 
process  in  which  staff  participates 

Solicits  input  from  p-t  staff , advisory 
board,  learners,  and  community 

Has  an  active /committed  advisory  board 
which  reflects  community 

Has  fair  distribution  of  administrative 
tasks 

Has  adequate  communication  and 
coordination  among  staff 

Devotes  appropriate  amount  of  staff  time 
to  administrative  tasks 

comments ; 
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EVALUATION 

conducts  on-going  program  evaluation 
and  review  of  program  outcomes 

Uses  evaluation  results  to  revise/refine 
program  design  and  assist  in  future 
planning 

Involves  staff  and  learners,  and  solicits 
input  from  advisory  board  and  community 

Demonstrates  ability  to  look  critically 
at  program  philosophy  and  goals 

Makes  evaluation  available  to  staff, 
learners  and  advisory  counsel 

Comments ; 


O 

4 


TRACKING/ REPORTING 


Strengthl  Need 
ESL  ABE  ESL  ASBE 


NA 

ESL  ABE 


Tracks /documents  learner 
progress 

Tracks  community  demographics, 
applications,  enrollments, 
attendance,  outcomes, and  waiting 
lists 

Uses  information  for  program  revision 
Comments : 


FISCAL  & PROGRAM  MANAGEMENT 

Tracks  budgets  by  funding  source 

Maintains  adequate  and  accurate 
fiscal  & statistical  record  systems 

Monitors  expenditures  against  budget 

Prepares  and  submits  accurate  and 
timely  reports 


Comments : 


INTERAGENCY  COORDINATION 

Has  formal  and  informal  interagency 
referral  agreements 

Links  staff  and  learners  with  other 
programs 

Participates  in  community  activities 


Comments : 
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FACILITIES 

Provides  good  learning/teaching 
enrivorunent 

Has  adequate  furnishings  and 
classroom,  office,  meeting, 
counseling,  and  resource  space 

Has  adequate  teaching  supplies 

Has  a working  photocopier 

Is  clean,  well-lit  and  ventilated, 
with  comfortable  temperatures 
year-round 

Is  handicap-accessible 
Comments ; 
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COMMUNITY  LEARNING  CENTER 

PROGRAM  EVALUATION  BY  STUDENTS 
1992 

CLASS  

TIME 

TEACHER  . 

Please  answer  all  the  questions  carefully. 

1.  How  long  have  you  been  coming  to  the  Learning  Center? 
years  months 


2.  We  want  to  know  how  the  school  has  helped  you..  Check  all  the 
things  you  learned  in  this  school.  Do  not  check  the  things  you 
learned  in  another  place. 


A.  Reading 


Did 

the 

school 

help 

yes 

you 

to 

understand  what  you  read? 
no 

Did 

the 

school 

help 

yes 

you 

to 

read  and  understand  a newspaper? 
no 

Did 

the 

school 

help 

yes 

you 

to 

enjoy  reading? 
no 

B.  Conversation 


Did  the  school  help  you  to  communicate  in  English? 

yes  no 

Did  the  school  help  you  to  understand  what  you  hear  on  the  radio  or 
TV? 


yes  no 
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C.  Math 


Do  you  have  a math  class  here? 

yes  no 

Did  the  school  help  you  learn  to  manage  your  money? 

yes  no 

Did  the  school  help  you  learn  to  fill  out  your  income  tax  forms? 
yes  no 

Did  the  school  help  you  learn  to  estimate  store  bills? 

yes  no 

Did  the  school  help  you  to  enjoy  doing  math? 

yes  no 

Did  the  school  help  you  learn  to  use  a calculator? 

yes  no 

D.  Jobs 

Do  you  have  a job? 

yes  no 

Did  your  classes  help  you  to  get  a job? 

yes  no 

Did  your  classes  help  you  to  get  a better  job  or  promotion? 

yes  no 


E.  Family 

Do  you  have  children? 

yes  no 

Did  your  classes  help  you  to  help  your  children  with  homework? 
yes  no 

Did  your  classes  help  you  to  read  to  your  children? 

yes  no 


F.  Community 

Did  the  school  help  you  to  help  other  people  in  your  family  or 
community? 

yes  no 

Did  the  school  help  you  to  make  friends  in  school  or  outside? 
yes  no 

Did  your  classes  help  you  to  get  a driver's  license? 

yes  no 


Did  the  school  help  you  to  get  a library  card? 
yes  no 


G.  Other 

Did  the  school  help  you  to  feel  more  confident  in  yourself? 
yes  no 

Did  you  learn  to  use  a computer  here? 

yes  no 

Did  you  learn  to  type  here? 

yes  no 


H.  Please  give  other  examples  of  what  you  learned  in  school  and  how 
the  school  has  helped  you. 


3.  What  do  you  like  best  about  the  school? 


4.  What  is  not  good  about  the  school?  How  can  we  make  the  school 
better? 


5.  Do  you  have  anything  else  to  tell  us? 
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QCC  INDIVIDUAL  INTERVIEW 


Appendix  5 


BACKGROUND  CONVERSATION 

WHERE  ARE  YOU  FROM? 

HOW  LONG  HAVE  YOU  BEEN  IN  THIS  COUNTRY?  

DID  YOU  COME  DIRECTLY  TO  THE  WORCESTER  AREA?  

DO  YOU  ;HAVE  FAMILY  HERE?  ' 

CHILDREN AGES  

EMPLOYMENT 

DID  YOU  WORK  IN  YOUR  COUNTRY? WHAT  KIND  OF  WORK? 

WHAT  KIND  OF  WORK  DO  YOU  DO  AT  - . 

CAN  YOU  DESCRIBE  SPECIFICALLY,  YOUR  JOB?  


EDUCATION 

Did  you  go  to  school  in  your  country? How  many  years?. 

What  was  school  like  in  your  country?  Did  you  enjoy  it?  

Have  you  ever  gone  to  school  in  the  United  States?  __ 

Are  you  teaching  your  children  your  own  native  language? 

Have  you  ever  taught  anything  else,  like  sewing,  cooking, 
driving,  sports?  

What  made  you  decide  that  you'd  like  to  come  to  classes  now? 
Do  you  know  specifically  what  you'd  like  to  learn? 


How  do  you  think  learning  to  read  or  write  better  will  help  you 
in  life? ■ 


What  does  your  family  think  of  this?  — 

What  do  you  do  when  you  have  trouble  reading  or  writing  something? 
(Does  someone  help  the  person?) 

at  home 

at  work  ■ 

LANGUAGE 

Do  you  read  or  write  in  your  own  language?  

Is  this  easy  or  difficult  for  you? 

What  do  you  read  in  English.  . . . the  phone  book 

medicine  bottles grocery  ads^ menus 

newspaper magazines 
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What  do  you  have  to  read  at  work? 

What  do  you  already  know  how  to  write? 

checks note  to  school 

application  forms things  at  work 

please  fill  out  this  information 

NAME 

ADDRESS 

( street) 

PHONE  NUMBER 

DEPARTMENT 


(city)  (state)  (zip! 

SHIFT 


Choose  one  writing  topic.  Write  three  or  four  sentences 

(1)  A note  telling  your  child's  teacher  that  he  or  she  is  sick  and 
cannot  come  to  school. 

(2)  Phone  message  for  your  wife/husband/friend. 

(3)  A post  card  to  a friend  while  you  are  on  a trip. 

(4)  A letter  to  your  landlord  complaining  about  something. 

(5)  A typical  work  memo. 
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READING 


CHOOSE  A READING  PASSAGE 
READING  SAMPLE  TITLE 

WHAT  WAS  THIS  PASSAGE  ABOUT? 


WAS  THIS  EASY  OR  DIFFICULT  FOR  YOU  TO  READ?  

IF  A SECOND  SELECTION  IS  MADE,  ASK  SAME  QUESTIONS 

IS  THERE  ANYTHING  YOU  DID  WHILE  YOU  WERE  READING  TO  HELP  YOU  IN 
TRYING  TO  UNDERSTAND  THIS  PASSAGE? 


IF  THERE  WERE  3 LEVELS  OF  CLASSES,  WHERE  WOULD  YOU  BE  MOST 
COMFORTABLE?  

ARE  THERE  ANY  SCHEDULING  PROBLEMS  WE  CAN  ASSIST  YOU  WITH? 
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School  History 


a.  What  are  your  strengths  and  weaknesses  in  reading?  Writing?  Arithmetic? 


b.  How  did  you  do  in  elementary  school?  Grades  7-8?  High  school? 


c.  Were  you  in  any  special  program  such  as  resource  room.  Title  I,  advanced 
class? 


d.  What  do  you  want  to  accomplish  in  our  class? 


Appendix  6 


One  Stop  

Futures  

School  Only  

JUST  A START  PROGRAM 
EDUCATION  PROGRAM  INTAKE/ REG I STRATI  ON 

Name . Oate 

Address ___  Telephone 

Date  of  8irth Age 

Student's  First  Language Literate  in  first  language?  Y N 

Educational  Background  and  Objectives 

Last  School  Attended_ 

City,  State Last  Grade  Attended 

Last  Grade  Completed High  School  Grad?  Yes(  ) No(  ) 

Date  of  Last  Attendance 


I. 

Self-rating  on  reading  skills: 

Excellent 

Good 

Fa  i r 

Needs  Work 

2. 

Self-rating  on  arithmetic: 

Excel  lent 

Good 

Fa  i r 

Needs  work 

3. 

Self-rating  on  writing  skills: 

Excellent 

Good 

Fa  i r 

Needs  work 

2 54 
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SRA 


) G/2) 


-.ssess.T.ent 


Reading  Comprehension  (G/l)  

T.-.BE 

Locator  test  Vocabulary  Math Total 

Complete  battery  level(s)  


Form  5 Form  5 


1. 

Vocabulary 

1 , 

Vocabulary 

2. 

Comprehension 

2 , 

Comprehension 

Total  Reading 

Total  Reading 

3. 

Math  Comp. 

3 . 

Math  Comp. 

4. 

Math  C&A 

•t 

~t  . 

Math  C&A 

Total  Math 

Total  Math 

5. 

Lang.  Mech. 

5 . 

Lang.  Mech. 

6. 

Lang.  Expr. 

6 . 

Lang.  Expr. 

Total  Lang. 

Total  Lang . 

Total  Battery 

Total  Battery 

7.  ' 

Spelling 

7 . 

. Soelling 

Writing  Samole 

Pre 

?o 

St 

G^-D  Testing 

Writing  

Social  Studies  

Science  

Literature 

Mathematics 
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STUDENT  LEARNING  GOALS 


Name 


Date 


Confidential 


2 56 


O 

ERIC 


69 


1 

Personal  Improvement! 

1 

I- 

would  like  to  1 
do  this  1 

I would  like  to  get  better  at: 

1 

note  takinq 

Yes 

studyinq 

Yes 

takinq  tests 

Yes 

yp  c 

following  directions 

Yes 

asking  for  help 

Yes 

organizing  my  work 

Yes 

planninq  my  time 

Yes 

working  with  others 

Yes 

solving  problems 

Yes 

finishing  what  I start 

Yes 

speaking 

Yes 

becoming  more  responsible 

Y e s 

Yes 

handwritinq 

Yes 

vocabulary 

Yes 

I 

would  like  to: 

register  to  vote 

Yes 

qet  a library  card 

Yes 

learn  to  use  the  library 

Yes 

Yes 

join  a community  organization  like  the  "Y" 

Yes 

get  my  G.E.D. 

Yes 

get  a hiqh  school  diploma 

Yes 

qo  to  colleqe 

Yes 

Yes 

start  a career 

-yes 

Please  list  other  "Personal  Improvement"  areas  you  would  like  to  work  on 


^ub^ect^^n^To^^s^ 


I know 

very 

I know 

1 

little 

about 

something 

this 

about  this 

J 

p want  to 
learn  aboutl 
[this 
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I kr 
iitt 

this 

lOW 

:le 

very 

about 

I k 
som 
a bo 

oow 

leth 

ut 

ing 

this 

I w« 
leat 
this 

mt 

:n  , 

to 

about 

weather 

the  human  body 

staying  healthy 

nutrition  (eating  healthy) 

nature 

animals/animal  behavior 

_enginee^r ing  (how  machines  work) 

Please  list  other  subjects  and  topics  you  would  like  to  learn  about. 
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EVERYDAY 
READING  SKILLS 


J^Know^Jtow 
To  Do  This 


^Do  This  All 

Zhe  Time 


I Would  T.ilra  Tr 


and  write  my  name 

Yes 

No 

Yes 

No 

Yes 

and  write  my  address 

Yes 

No 

Yes 

No 

Yes 

and  write  my  phone  number 

Yes 

No 

Yes 

No 

Yes 

and  write  my  social  security  # 

Yes 

No 

Yes 

No 

Yes 

and  write  checks 

Ye  s 

No 

Yes 

No 

Yes 

and  write  letters 

Ye  s 

No 

Yes 

No 

Yes 

and  fill  out  forms 

Yes 

No 

Yes 

No 

Yes 

and  use  a dictionary 

Yes 

No 

Yes 

No 

Yes 

labels  & instructions  on 

, food 

Yes 

No 

Yes 

No 

Yes 

1 medicine 

Yes 

No 

- Yes 

No 

Yes 

clothes 

Yes 

No 

Yes 

No 

Yes 

J applicances 

Yes 

No 

Yes 

No 

Yes 

a calendar 

Y e s 

No 

Yes 

No 

Yes 

| a bus  schedule 

Yes 

No 

Yes 

No 

Yes 

t . v . guide 

Y e s 

No 

Yes 

No 

Yes 

| a phone  book 

Yes 

No 

Yes 

No 

Yes 

1 menus/recipes 

Yes 

No 

Yes 

No 

Yes 

.bills 

Yes 

No 

Yes 

No 

Yes 

1 maps 

Yes 

No 

Yes 

No 

Yes 

newspapers 

i 

Yes 

No 

Yes 

No 

Yes 

| magazines 

Yes 

No 

Yes 

No 

Yes 

apartment  lease 

Yes 

No 

Yes 

No 

Yes 

Jwant  Advertiser 

Yes 

No 

Yes 

No 

Yes 

lonsumer  Reports 

Yes 

No 

Yes 

No 

Yes 

jstreet  signs 

Yes 

No 

Yes 

No 

Yes 

'health  information 

Yes 

No 

Yes 

No. 

Yes 

.-a  map 

Yes 

No 

Yes 

No 

Yes 

Notices  from,  welfare 

office,  community 

Yes 

No 

Yes 

No 

Yes 

|to  my  children 

Yes 

No 

Yes 

No 

Yes 

otes  from  my  child’s  school 

Yes 

No 

Yes 

No 

Yes 

PAR  • gmflVMgHT: 


dventures 

Yes 

No 

Yes 

No 

Yes  1 

mysteries 

Yes 

No 

Yes 

No 

Yes 

;>mance 

Yes 

No 

Yes 

No 

Yes 

..istory 

Yes 

No 

Yes 

No 

Yes 

ERLC 
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EVERYDAY 
MA'l'k  SXTTlS 


I Know  How 

To""po*"TK7T 


writing  dates  as  nuraoers  yes  No 

telling  time  Yes  ~No‘ 

writing  the  time  " Yes  “ 

understanding  a batting  average  yes  No" 

add/suDstract  money 
count  change 

estimate  cost  of  groceries 

savTngs"^TtH^aTe^^^^oupons 
rind  lowest  priced  groceries 
con^re^osF^J™me3TcTnes 
reaa  tnermometer 
usecro!cTng"measures 
find  total  calor i es 
.read  nutrition  information 
learn  to  measure 

[ iearn  metricsystem"of*Tieasurenient 
finding  car  mileage 
| total  a restaurant  check 
’figure  sales  tax 
•figure  tipping 
Iread  sales  tax  chart 
nake  a budget 
| set  savings  goal 
:ind  annual  car"expenses 
[reading  a bill 
figure  cost  of 
jrigure  credit  card  charges 
^icaTate^nterest^TTnOTey" 

• -stir!fite""cost"" o^"Eom^ 

• repair  and  improvement 
ind  dimensions  for  scale 

| figure  amount  of  materials 
sing  a checking  account 
using  a calculator 

Lease  list  other  "Everyday  Math  Skills"  you  would  be 


I Do  This  All 
The  Time 


I Would  Li  Ira  «T.^ 


Yes 

> Yes* 

No 

No 

Yes 

Yes 

No 

~ No 

Yes 

Yes 

Yes" 

Yes" 

No 

No 

~ Yes 

Yes" 

~ No 

No 

Yes 

Yes 

Yes" 

No 

Yes 

No 

Yes  | 

Yes 

Ye  s 

No 
No 
No 

Yes 

No 

Yes  1 

Yes 

Yes 
Yes 

No 
No 

__Yes_1 

Yes  I 

Yes 

No 

Yes 

No 

• Yes 

No 

Yes 

No 

Yes""™] 

Yes 

No 

Yes 

No 

Yes  1 

Yes 

No 

Yes 

No 

Yes  | 

Yes 

No 

Yes 

No 

Yes  | 

Yes 

No 

Yes 

No 

Yes  1 

Yes 


No 


Yes 


No 


Yes 


_Yes 

~Yes 

_Yes 

No 

Yes 

wo 

No 

^Yes 
Yes 

No 

_No 

Yes 

Ye  $ 

No 

No 

Yes 

Yes 

_Yes 

No 

Yes 

No 

Yes 

_Yes 

No 

Yes 

No 

^^__Yes 

_Yes 

No 

Yes 

No 

Yes 

_Yes 

No 

Yes 

No 

Yes 

_Yes 

No 

Yes 

No 

_^^^Yes 

Yes 

No 

Yes 

No 

Yes 

Yes 

No 

Yes 

No 

Yes 

Yes 

No 

Yes 

No 

Yes 

interested  in  learning. 


jSCHOOL^MATHJ 


IJCnowHov/ 
To  Do  This 


— Do  This  A11  I Would  r.ii,f  Tr 


addition 

Yes 

No 

Yes 

NO 

Yes 

subtraction 

Yes 

No 

Yes 

No 

Yes 

multiplication 

Yes 

No 

Yes 

No 

Yes 

division 

Yes 

No 

Yes 

No 

Yes 

weights/measures 

Yes 

No 

Yes 

No 

Yes 

measurement 

Yes 

No 

Yes 

No 

Yes 

decimals 

Yes 

No 

Yes 

No 

Yes 

fractions 

Yes 

No 

Yes 

No 

Yes 

ratio/proportions 

Yes 

No 

Yes 

No 

Yes 

percent 

Yes 

No 

Yes 

No 

Yes 

word  problems 

Yes 

No 

Yes 

No 

Yes 

algebra 

Yes 

No 

Yes 

No 

Yes 

geometry 

Yes 

No 

Yes 

No 

Yes 

Pleise  list  other  "School  Math"  skills  you  would  be  interested  in  learnin. 


[writing  skills  I 


using  capital  letters 

Yes 

No 

Yes 

No 

Yes 

punctuation 

Yes 

No 

Yes 

No 

Yes 

spelling 

Yes 

No 

Yes 

No 

Yes 

parts  of  speech 

Yes 

No 

Yes 

No 

Yes 

grammar 

Yes 

No 

Yes 

No 

Yes 

writing  telephone  messages 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

writing  checks 

Yes 

No 

Yes 

No 

Yes 

Yes 

No 

Yes 

No 

Yes 

things  to  do  list 

Yes 

No 

Yes 

No 

Yes 

a shopping  list 

Yes 

No 

Yes 

No 

Yes 

Yes 

No 

Yes 

No 

Yes 

letters  to  friends  & relatives 

Yes 

No 

Yes 

No 

Yes 

assistance 

Yes 

No 

Yes 

No 

Yes 

Yes 

No 

Yes 

No 

Yes 

stories 

Yes 

_ No 

Yes 

No 

Yes 
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Please  list  other  "Work  Skills"  you  would  like  to  learn 
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One  Stop 

Futures 

Evening  Class 


JUST  A START 


EDUCATIONAL  GOALS  PLAN  AND  PROGRESS  RECORD 


Student  

Date  enrolled 

Class 


Initial  Planning  Session:  Date 

Long  range  educational  goals: 


Teacher 


Long  range  employment  goals: 


Student : 
Teacher : 
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EDUCATIONAL  GOALS 


Date 


Name 


Period 


7.  Work  Skills 


Planning  Session: 


Date 


Teache 


Accomplishment  of  goals  from  previous  period 


Vocational  goals 


Student: 

Teacher : 

Counsellor : 


Final  Planning  and  Assessment 


Date 


Teacher 


Accomplishment  of  goals  from  previous  pprinH 


Progress  made  during  period  of  enronmpnh 


Next  steps 


Student: 

Teacher : 

Counsellor: 
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Adventures 


sitient 


Learner-centered  approaches 
to  assessment  and  evaluation 
in  adult  literacy  ><2 

2-  ' 2.2  Volume  4 22223? 

April  1993 
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MS  ABES 

f ^®v\\  Systam  ter  Adul  Basic  Eduction  Suopcrt  j 
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SABES 

SyszamtorAdDl  Btslc  Education  Support 


SABES  is  the  System  for  Adult  Basic  Education  Support,  a comprehensive 

training  and  technical  assistance  initiative  for  adult  literacy  educators 

programs.  Its  goal  is  to  strengthen  and  enhance  literacy  services  and  thus  to  ' * ' ,f 

enable  adult  learners  to  attain  literacy  skills.  •-  • 


worishops,  consultation,  mini-courses,  mentoring  and  peer  coaMng,  atidM^S 
other  training  activities  provided  by  five  Regional  Support  Centers  located  at 
community  colleges  throughout  Massachusetts.  SABES  also  offers  a 15-hour 
Orientation  that  introduces  new  staff  to  adult  education  theory  and  practice 
and  enables  them  to  build  support  networks. 


SABES  also  maintains  an  adult  literacy  Clearinghouse  io  coiieci,  evaluate,  and 
disseminate  ABE  materials,  curricula,  methodologies,  and  program  models, 
and  encourages  the  development  and  use  of  practitioner  and  learner-generated. : 
materials.  Each  of  the  five  SABES  Regional  Support  Centers  similarly  offers ' " 
program  support  and  a lending  library.  SABES  maintains  an  Adult  Literacy 
Hotline,  a statewide  referral  service  whch  responds  to  calls  from  new  learners : ; 
and  volunteers.  The  Hotline  number  is  1-800-447-8844. 

& 


The  SABES  Central  Resource  Center,  a program  of  World  Education,  pub- 
lishes a statewide  quarterly  newsletter,  "Bright  Ideas,"  and  journals  on  topics 
of  interest  to  adult  literacy  professionals,  such  as  this  volume  of  "Adventures 
in  Assessment."  ... 

The  first  three  volumes  of  "Adventures  in  Assessment"  present  a comprehen- 
sive view  of  the  state  of  practice  in  Massachusetts  through  articles  written  by 
adult  literacy  practitioners.  Volume  1,  Getting  Started,  includes  start-up 
and  intake  activities;  Volume  2,  Ongoing,  shares  tools  for  ongoing  assessment 
as  part  of  the  learning  process;  Volume  3,  Looking  Back,  Starting  Again,"", 
focuses  on  tools  and  procedures  used  at  the  end  of  a cycle  or  term,  including ; . 
self,  class,  and  group  evaluation  by  both  teachers  and  learners.  Volume  4 and 
future  issues  will  either  be  dedicated  to  specific  topics  or  a range  of  interests. 

"Adventures  in  Assessment"  is  free  to  Massachusetts  programs;  out-of-state 
requests  will  be  charged  a nominal  fee,  at  cost.  Please  write  to,  or  call : 


■ ■ v ■ 


Elizabeth  Santiago 
SABES  Central  Resource  Center 
World  Education 
210  Lincoln  Street 
Boston,  MA  02111 
617-482-9485 
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Introduction 

Looking  Back 

How  to  begin?  Where  to  begin? 

I read  back  through  my  other 
carefully  composed  introductions  for 
help.  Maybe  I said  something  I can  use 
to  get  me  going.  No  luck.  There  is  no 
neat,  overriding  category  that  I can  use 
to  frame  the  articles  I received  for  this 
volume  of  Adventures  in  Assessment. 
The  categories  or  components:  Getting 
Started,  Ongoing,  and  Looking  Back 
still  provide  a useful  framework,  but 
I'm  growing  bored  talking  about  tools, 
checklists,  questionnaires,  and  surveys. 

I want  to  talk  with  you  on  another 
level,  the  level  of  why.  I want  to  talk  not 
from  the  perspective  of  what  researchers 
are  saying  about  what  learner-centered 
approaches  to  assessment  are,  but  what 
you  and  I know.  The  authors  in  these 
pages  are  telling  us  about  the  teaching/ 
learning  process  and  how  assessment 
either  helps  or  hinders  this  understand- 
ing. If  we  believe  in  the  validity  of 
teachers  as  researchers  of  their  own 
practice,  it  is  time  for  us  to  really  listen 
by  to  the  voices  of  these  narratives  and  not 

Loren  McGrail  just  focus  on  the  tools  they  have  devel- 
editor  oped,  adapted,  or  modified. 

As  I was  reading  through  the  articles 
for  this  volume  again,  for  pleasure  now, 
and  not  for  editing  purposes,  a few 
voices  sang  out  to  me  loud  and  clear. 
They  said,  "underline  this,  this  is  impor- 
tant, you  could  build  an  introduction 
around  this."  So,  colleagues,  I would  like 
to  change  the  format  and  the  nature  of 
what  an  introduction  is  supposed  to  do. 


and  tell  you  instead  some  of  the  things  I 
learned  from  listening  to  what  the 
practitioners  and  learners  in  this  volume 
have  to  say. 

The  voices  from  the  learners  Nancy 
Venator  interviewed  in  Through  the 
Eyes  of  an  ABE  Interviewer  stand  out 
immediately.  "When  I could  not  read 
and  went  into  stores,  there  was  an 
animal  inside  of  me  (he  touched  his 
stomach).  As  I learned  more  and  more 
the  animal  slowly  went  away  (he  moved 
his  hand  slowly  up  to  his  chest)  and  now 
it  is  gone  forever  (he  moved  his  hand 
upward  and  out  with  a flourish)  [page 
44]."  Hearing  the  way  learners  actually 
talk  about  their  own  learning  process  is 
a baseline  for  me  by  which  to  measure 
all  that  we  say  and  do.  The  "animal 
inside"  reminds  me  of  how  little  I really 
know  about  just  what  it  feels  like  not  to 
be  literate. 

I do  know,  however,  what  Dulany 
Alexander  is  talking  about  in  her  article. 
The  ESL  Classroom  as  Community: 
How  Self-Assessment  Can  Work,  when 
she  writes,  "The  kind  of  evaluation, 
defined  as  progress  from  one  level  to  the 
next,  has  meant  that  if  learners  don't 
keep  progressing,  we  don't  get  paid 
[page  34]."  Her  voice  has  a kind  of 
resignation  that  reminds  me  of  Sally 
Spencer's  words  when  she  remarks, 
"Passing  the  GED  test,  which  still  tests 
for  'single  correct  responses',  is  the 
primary  goal  of  the  GED  student  [page 
42]."  I can  imagine  Eileen  Barry  joining 
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this  duet  and  adding  her  own  complaint 
about  the  TABE:  "After  the  students 
completed  the  assessment,  it  was  our 
turn  to  struggle  with  it.  We  felt  frus- 
trated as  we  looked  at  the  answers,  both 
correct  and  incorrect,  because  we  had  no 
way  of  knowing  why  the  student  chose  a 
particular  answer  [page  22]." 

Missing  from  this  chorus,  however,  is 
the  Even  Start  project  in  Amesbury, 
which  is  mandated  to  use  the  CASAS  for 
their  formalized  assessment  process. 
They  seem  to  have  managed  to  include 
their  CASAS  assessment  in  their  more 
holistic  intake  process.  I still  wonder 
how  the  CASAS  results  are  integrated 
into  the  adult  learner's  individualized 
learning  plan.  Or  why  they  needed  to 
develop  another  goalsetting  tool  beyond 
the  CASAS.  Is  it  because  their  goal  sheet 
offers  elements  of  "affective  mea- 
surement," a chance  for  parents/adult 
learners  to  express  their  dreams  and 
wishes,  combined  with  "felt"  or  "real" 
strengths  and  weaknesses  which  can 
later  provide  learners  with  an  opportu- 
nity to  "ethically  analyze  how  they 
perceive  themselves,  and  whether  their 
perceptions  have  helped  or  hindered 
their  progress  [page  18]." 

An  aversion  to  standardized  measure- 
ments is  not  all  these  practitioners  have 


in  common  however.  In  addition  to 
sharing  a view  of  literacy  as  practice, 
how  learners  use  literacy  in  their  daily 
lives,  inside  and  outside  the  classroom, 
they  share  the  view  that  how  learners 
judge  their  own  capabilities  and  their 
own  progress  is  also  critical.  Many  of  the 
tools  they  have  developed,  whether  they 
are  tape  journals  (page  38),  informal 
reading  inventories  (page  22),  goal- 
setting activities  (pages  8 and  15),  or 
self-evaluation  forms  (page  34),  ask  the 
learners  to  redraw  their  image  of  them- 
selves as  learners  from  a position  of 
strength,  not  as  isolated  individuals,  but 
as  learners  "ioininp-  a community  of 
learners  who  share  similar  experiences, 
frustrations,  and  accomplishments  [page 
22].” 

These  are  my  reactions,  dear  reader, 
some  intellectual,  some  gut.  I invite  you 
to  practice  what  we  preach  about  read- 
ing not  being  a passive  activity,  to 
interact  with  these  articles,  to  ask  ques- 
tions, make  predictions,  look  for  answers 
and  — if  you  feel  like  sharing  your 
responses  — write  to  us  at  Adventures  in 
Assessment  so  we  can  share  your  ideas 
with  a growing  circle  of  practitioners 
committed  to  participatory  approaches 
to  assessment.  BT 


If  we  believe  in 
the  validity  of 
teachers  as 
researchers  of 
their  own  practice, 
it  is  time  for  us  to 
really  listen  to  the 
voices  of  these 

j 

narratives  and  not 
just  focus  on  the 
tools. 
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Group  Goal  Setting  Activities: 

An  Approach  from  Youth  Service  Corps 


excerpted  from  the 

PECE  Resource  and 
Planning  Guide 


for  more  information: 
Michael  Sack 
Clare  Ignatowski 
Public/Private  Ventures , 
Inc. 

399  Market  Street 
Philadelphia,  PA  19106 
215-592-9099 


This  article  is  excerpted  from  Chapter  3 of 
the  PECE  Resource  and  Planning  Guide. 
PECE,  Practical  Education  for  Citizenship 
and  Employment,  is  the  name  of  the  educa- 
tion component  developed  for  the  Urban 
Corps  Expansion  Project  (UCEP),  a three- 
year  demonstration  project  designed  by 
Public/Private  Ventures,  Inc.  of  Philadel- 
phia. 

UCEP  is  a combination  community  service/ 
education  model  now  operating  at  11  sites 
nationwide.  Each  autonomous  program 
serves  50-100  youths,  aged  17-23,  who  are 
unemployed  and  out  of  school.  Approxi- 
mately seventy- five  percent  do  not  have 
diplomas  or  GEDs. 

Corpsmembers  (as  program  participants  are 
referred  to)  are  given  paid  employment  in 
human  services  and  community  service 
projects,  such  as  rehabbing  buildings  for 
non-profit  developers,  in  addition  to  the 
program's  educational  component. 

The  manual  from  which  this  chapter  is  taken 
has  been  written  to  be  useful  outside  the 
urban  youth  service  corps,  such  as  in  sum- 
mer jobs  programs.  It  is  written  for  trainers 
and  educators. 

Chapter  3 focuses  on  setting  goals  and 
creating  learning  plans.  In  this  volume  of 
Adventures  in  Assessment,  we  focus  on 
group  goal-setting.  In  the  next  volume,  we 
will  continue  the  part  of  the  chapter  which 
discusses  individual  goal  setting. 


The  process  of  setting  goals  and 
planning  steps  to  achieve  them  is 
ongoing  throughout  each  corpsmem- 
ber's  experience  in  the  program.  The 
way  you  begin  this  process  with  new 
corpsmembers  has  a special  importance. 
Many  of  the  young  adults  who  enter  the 
corps  are  burdened  with  a history  of 
failure  in  school  and  a sense  of  education 
as  something  that  has  been  imposed  on 
them. 

But  most  also  enter  with  the  motiva- 
tion to  change  their  lives.  Your  early 
goal-setting  activities  are  an  opportunity 
to  build  on  this  motivation  and  help 
corpsmembers  begin  to  redraw  their 
image  of  themselves  as  learners  and  to 
rethink  what  learning  is  all  about. 
Through  these  activities,  corpsmembers 
should  discover  that  they  are  responsible 
for  defining  where  they  want  to  go  and 
what  they  have  to  do  to  get  there  — and 
that  you  are  going  to  support  them. 

SETTING  GOALS 
Goal  setting  is  really  an  exercise  in 
problem  solving.  Corpsmembers  have  to 
learn  to  ask  — and  try  to  answer  — a 
series  of  questions  that  enable  them  to 
define  their  choices  and  decide  how  to 
get  where  they  want  to  go.  Most  simply 
the  questions  are:  What  are  my  goals? 
What  must  I do  to  reach  my  goals?  To 
answer  these,  corpsmembers  must  work 
through  a series  of  more  precise  ques- 
tions: 
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>■  Where  do  I want  to  be  in  five  years 
(or  four  weeks,  or  six  months,  or  ten 
years)? 

>■  What  must  I know  to  get  there? 

>■  What  steps  must  I take  in  order  to 
know  and  be  able  to  do  these  things? 
>■  What  abilities  and  experience  do  I 
already  have  that  are  going  to  help 
me  take  these  steps? 

What  obstacles  might  be  in  my  way? 
>-  How  can  I deal  with  those  obstacles? 

>•  What  should  I do  first,  second  and  so 
on? 

These  are  tough  questions  for  anyone 
to  answer:  adults,  young  adults,  college 
students,  at-risk  youth.  But  they  are  also 
essential  questions  to  the  goal-setting 
process,  which  involves  defining  needs, 
evaluating  one's  own  strengths  and 
weaknesses,  and  planning  and  carrying 
out  strategies. 

Group  Activities 

Goal  setting  activities  that  take  place 
in  groups  during  orientation  or  in  the 
beginning  of  a program  should  help 
corpsmembers  identify  broad,  long-term 
needs  and  begin  to  see  them  in  relation 
to  what  they  can  accomplish  in  the 
corps.  In  almost  every  case,  corps- 
members  will  end  up  with  a product  — 
a questionnaire,  a story,  a timeline,  a 
goal  chart  — that  should  be  placed  in 
their  portfolio,  where  it  can  be  used  as  a 
starting  point  for  developing  personal 
learning  plans  during  your  one-on-one 
meetings. 

The  group  activities  lay  the  founda- 
tion for  the  next  step  in  the  process, 
when  corpsmembers  more  precisely 
define  their  goals  in  the  corps:  in  life 
skills  (employment,  community  partici- 


pation, personal  development), 
work,  and  personal  academics. 
The  group  activities  are: 


Option  #1:  Thinking  about  Learning. 
Corpsmembers  explore  their  past  experi- 
ences with  education  and  start  to  define 
their  learning  needs. 


Option  #2:  Guest  Speaker.  A speaker 
from  outside  the  corps  discusses  his  or 
her  experiences  overcoming  obstacles 
and  achieving  goals. 


Option  #3:  Goals  Questionnaire. 
Corpsmembers  identify  reasons  they 
joined  the  corps  and,  in  the  process. 


fKinV  ahnnf  nnfonti'al  rrAGlr 

Ultjr  llllglll 

not  have  previously  considered. 


Option  #4:  Creating  a Future.  Corps- 
members  make  a collage  or  write  a story 
that  describes  a potential  future  for 
themselves. 


Option  #5:  A Timeline.  Corpsmembers 
create  a timeline  in  order  to  be  more 
specific  about  their  goals  and  begin  to 
see  the  relationship  between  short-  and 
long-term  goals. 

Option  #6:  Goal-Setting  Chart.  In  this 
follow-up  activity  to  the  timeline, 
corpsmembers  look  at  the  relationship 
between  what  they  can  accomplish  in 
the  corps  and  their  longer-term  goals. 

Option  #7:  Returning  to  the  Corps  10 
Years  Later.  In  another  way  of  looking 
at  their  long-term  goals,  corpsmembers 
imagine  themselves  10  years  in  the 
future,  returning  to  the  Corps  either 
for  a reunion  or  as  a guest  speaker. 

[Editor's  note:  this  excerpt  includes 
Activities  1,2,5  & 7.] 
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The  group 
activities  lay  the 
foundation  for  the 
next  step  in  the 
process,  when 
corpsmembers 
more  precisely 
define  their  goals  in 
the  corps:  in  life 
skills  (employment, 
community 
participation, 
personal  develop- 
ment), work,  and 
personal  academics. 
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Group  Goal  Setting  Activities 
Option  # I : 

Thinking  About  Learning 

This  activity  allows  corpsmembers  to  explore  their  past  experiences  with  education:  what 
they  remember  as  positive  and  negative  learning  experiences  and  how  they  learn  best. 
Corpsmembers  also  explore  their  own  learning  needs:  what  makes  learning  comfortable  and 
what  makes  learning  uncomfortable  (and,  therefore,  difficult). 

Materials:  Photographs  and  illustrations,  readings,  newsprint 

1.  Start  by  having  corpsmembers  look  at  a variety  of  pictures  of  people  learning  something  in 
school  and  in  other  settings  (especially  work).  Include  people  who  are  happy,  miserable, 
young  and  old.  Have  corpsmembers  describe  the  pictures,  then  talk  about  the  memories 
they  evoke. 

2.  Read  aloud  a few  excerpts  from  descriptions  that  people  have  written  about  their  experi- 
ences learning  and  in  school.  After  each  excerpt,  let  corpsmembers  discuss  what  they 
have  heard:  How  does  it  connect  with  their  own  experiences?  Encourage  corpsmember 
to  tell  their  own  stories. 

3.  Divide  the  corpsmembers  into  two  groups,  and  ask  one  group  to  make  a list  of  phrases 

that  could  complete  the  fragment  “It’s  easy  to  learn  when ” Ask  the  other  group  to 

complete  "It’s  hard  to  learn  when....”  When  both  groups  are  done,  have  volunteers  copy 
the  groups  responses  onto  newsprint.  Here  are  some  possible  responses: 

It’s  easy  to  learn  when: 

The  teacher  is  good. 

We  pay  attention. 

People  aren’t  making  fun  of  each  other. 

We  know  why  we  are  learning  something. 

We  understand  what’s  going  on. 

It’s  hard  to  learn  when: 

The  teacher  talks  too  much. 

People  laugh  at  you. 

We  don’t  understand. 

We  have  other  things  on  our  mind. 

The  teacher  thinks  we’re  stupid. 
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4.  Follow  up  these  lists  by  having  the  whole  group  discuss  some  items  more  specifically. 
With  the  items  here,  for  example,  you  can  ask  what  makes  a good  teacher,  what  makes  it 
hard  to  understand  something,  or  what  they  can  do  to  have  a group  where  people  aren't 
making  fun  of  each  other.  If  the.  corpsmembers  have  begun  keeping  journals,  suggest 
possible  journal  topics.  For  example: 

An  experience  (in  school  or  on  the  job)  where  they  felt  good  about  learning  something: 
What  did  they  learn?  Why  did  it  make  them  feel  so  good? 

• A description  of  someone  from  whom  they  learned  (a  teacher,  an  employer,  a co- 
worker, anyone). 

• A description  of  a time  when  someone  learned  something  from  them. 

• An  experience  at  school  or  work  that  made  it  hard  for  them  to  learn. 

You  might  want  to  allow  time  for  corpsmembers  to  write  during  the  activity.  Then  those 
who  want  to  can  read  their  pieces  aloud. 


[Adapted  from  Andy  Nash  in  Elsa  Auerbach,  Making  Meaning,  Making  Change:  A Guide  to  Participatory 
Curriculum  Development  for  Adult  ESL  and  Family  Literacy.  University  of  Massachusetts  English  Family 
Literacy  Project,  Boston,  1 990] 
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Group  Goal-Setting  Activities 
Option  #2: 

Guest  Speaker 

This  activity  helps  corpsmembers  think  about  the  process  of  setting  and  achieving  goals  by 
listening  to  someone  else  speak  about  his  or  her  experience.  The  speaker  should  be  someone 
the  corpsmembers  can  see  as  a role  model. 

1 . Invite  a guest  speaker  from  the  community  who  has  “made  it”:  who  has  set  and  achieved 
goals  for  herself  or  himself.  Prepare  the  guest  speaker  by  giving  him/her  some  background 
about  the  corps’s  goals,  and  some  idea  where  the  corpsmembers  are  in  their  own  goal 
setting  process.  Ask  the  speaker  to  talk  about  his  or  her  own  goal  setting  process  and 
strategies:  the  goals  that  he  or  she  has  set  (and  why),  the  steps  needed  to  achieve  those 
goals,  the  difficulties  encountered  and  how  he  or  she  overcame  the  difficulties.  The 
speaker  may  be  able  to  include  comments  on  values,  self  esteem,  sex-role  stereotypes, 
and  decision  making  as  well  as  goal  setting. 

2.  Let  corpsmembers  know  in  advance  that  there  will  be  a speaker  and  what  he/she  will  be 
talking  about.  Corpsmembers  should  have  a sense  of  what  to  listen  for  and  how  the  topic 
relates  to  their  own  lives.  They  should  also  think  about  questions  they  might  want  to  ask. 

3.  After  the  presentation,  allow  time  for  discussion  and  questions  from  corpsmembers. 
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Group  Goal-Setting  Activities 
Option  #5: 

A Timeline 

This  activity  encourages  corpsmembers  to  become  more  specific  about  their  goals  and  begin 
to  see  the  relationship  between  short-term  and  long-term  goals. 

Materials:  Newsprint  (for  you  to  draw  your  timeline  on  as  you  model  the  steps  in  this  activity),  paper, 
pens  or  pencils. 

1.  Have  corpsmembers  draw  the  first  part  of  their  timelines:  from  birth  to  present  They 
should  write  in  dates  and  major  events  that  have  happened  in  their  lives  (you  will  want 
to  model  this  and  the  next  two  steps  on  newsprint). 

2.  Now  have  the  corpsmember  extend  their  timelines  two  years  into  the  future  and  write 
in  events  they  want  to  make  happen  in  those  years  (this  will  include  in  and  beyond  the 
corps,  such  as  earning  their  GED,  graduating  from  the  corps,  joining  the  Army,  getting 
married,  starting  college,  getting  job  training,  buying  a car,  etc.). 

3.  Finally,  have  them  extend  their  timelines  to  reach  about  ten  more  years  into  the  future. 
Ask  them  to  add  the  events  they  would  like  to  experience  during  this  period  (they  might 
include  such  goals  as  living  in  a place  of  their  own,  getting  promoted,  earning  a college 
degree,  taking  a trip,  getting  married,  etc.). 

4.  Corpsmembers  may  want  to  display  their  timelines  in  the  room  or  hold  them  up  to  share 
with  the  group.  Stimulate  discussion  by  asking  questions  like  the  following: 

How  much  control  do  people  have  over  what  happens  in  the  early  years  of  their  lives  — 
up  to  age  1 0 for  example?  What  about  during  the  second  ten  years:  do  we  get  to  set  goals 
for  ourselves  and  work  toward  them  in  our  teen  years?  What  about  when  we’re  in  our 
20s  or  30s? 

What  connections  do  you  see  between  your  goals  in  the  next  two  years  and  your  goals 
for  the  ten  years  after  that? 
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Group  Goal-Setting  Activities 
Option  #7: 

Returning  to  the  Corps  Ten  Years  Later 

This  activity  provides  another  way  for  corpsmembers  to  think  about  their  long-term  goals: 
by  imagining  themselves  ten  years  in  the  future,  returning  to  the  corps  either  for  a reunion  or 
as  a guest  speaker.  In  the  first  scenario,  corpsmembers  fill  out  a questionnaire;  in  the  second 
scenario,  they  prepare  a short  speech,  which  they  can  present  to  the  rest  of  the  group. 

Allow  corpsmembers  to  select  the  choice  they  feel  more  comfortable  with. 

A.  Corps  Reunion:  It’s  been  ten  years  since  you  were  in  the  corps,  and  now  you’ve  been 
invited  back  for  a reunion.  The  corps  has  sent  you  a questionnaire  to  fill  out  about  what 
you’re  doing  now.  They  are  going  to  make  all  of  the  questionnaires  into  a booklet,  which 
they  will  give  to  everyone  that  comes  to  the  reunion. 

I • I am years  old.  (Remember,  this  is  ten  yean  into  the  future.) 

2.  My  job  is: 

3.  My  responsibilities  at  work  include: 

4.  After  I left  the  corps,  I prepared  for  this  job  by: 

5.  My  family  responsibilities  are: 

6.  My  most  important  personal  possessions  are: 

7.  Of  my  experiences  in  the  last  ten  years,  these  have  been  the  best  ones: 

B.  Returning  to  the  corps  as  a guest  speaker:  You’ve  worked  hard  and  been  a big  success  since 
you  graduated  from  the  corps  ten  years  ago.  Now  the  corps  director  has  invited  you  back 
to  speak  to  corpsmembers  about  how  you  set  and  achieved  your  goals.  She/he  asks  you 
to  talk  about  what  you  have  done  in  the  ten  years  since  you  graduated. 

You  want  to  write  down  notes  to  prepare  your  talk  and  this  is  how  you  organize  them: 

• The  goals  I set  for  myself  and  the  work  I'm  doing  now: 

• The  steps  I took  to  achieve  my  goals: 

• Difficulties  I encountered: 

• Things  I did  to  overcome  those  difficulties: 

• My  goals  for  the  future  and  how  I plan  to  achieve  them: 

Write  down  the  notes  for  your  talk.  Then,  if  you  wish,  give  the  talk  to  other  people  in 
your  group. 
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The  Even  Start  Model 

Empowering  the  Student 
through  Goal  Setting 


The  North  Shore  town  of  Amesbury, 
Massachusetts  is  serving  as  a site 
of  Even  Start,  a federally  funded  model 
addressing  the  dual  issues  of  parenting 
and  literacy. 

The  four-year  grant  from  the  U.S. 
Department  of  Education,  awarded  to 
the  Amesbury  Public  School  system  to 
service  48  target  families  has  two  pri- 
mary eoals:  surmortine  adult  parenting 
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skills  and  empowering  parents  as  their 
children's  first  teacher  by  developing 
literacy  skills. 

Amesbury  Even  Start  is  based  upon  a 
case-management  model.  Each  family 
entering  the  program  is  assigned  a 
Home-Based  Visitor  and  a Family 
Literacy  Specialist.  The  interrelated 
nature  of  the  program  is  emphasized  by 
including  the  Family  Literacy  Specialist, 
the  Home-Based  Visitor,  and  the  parent 
in  the  development  of  goals  for  the 
family  in  each  component:  adult  educa- 
tion, early  childhood  education,  and 
parenting  family  issues.  These  goals  are 
then  used  to  plan  that  family's  involve- 
ment in  Even  Start  activities. 

The  program  is  organized  in  eight- 
week  service  cycles.  At  the  beginning  of 
each  cycle,  families  set  short-term  and 
long-term  goals  for  themselves  and  their 
children.  Then  staff  collaborate  with 
parents  in  the  selection  of  activities  to 
attain  individual  goals.  At  the  end  of  the 
eight-week  cycle,  the  staff  meet  again 
with  the  parent  to  evaluate  progress  and 
possibly  to  reassess  and  set  new  goals. 


THE  ADULT  LITERACY  COMPONENT 

The  adult  literacy  component  of  Even 
Start  is  a multi-level  program  aimed  at 
increasing  basic  skills,  improving  self- 
concepts,  reinforcing  various  uses  of 
literacy,  maximizing  parent  involvement 
in  child-centered  learning,  and  creating 
awareness  of  career/ employment  oppor- 
tunities. 

At  the  most  basic  level,  the  nroeram 
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provides  an  on-site  adult  literacy  pro- 
gram focusing  on  attaining  functional 
and  GED  skills.  At  another  level,  parents 
are  learning  about  child  development 
in  order  to  actively  participate  in  their 
children's  learning. 

We  have  recently  incorporated  the 
use  of  small  groups  into  our  literacy 
program.  These  small  groups  are 
planned  by  staff  based  on  the  general 
goals  of  the  adult  learners.  The  parents 
can  "opt"  into  any  one  of  these  groups 
at  the  beginning  of  a cycle  if  relevant. 

The  groups  have  helped  build  a sense 
of  community  within  the  program  and 
many  friendly  relationships  have 
formed. 

For  the  adult  learner  who  is  consid- 
ered a lower-level  ABE  student,  one-on- 
one  tutoring  is  provided.  When  the 
parent  feels  he/she  is  ready,  he/she  can 
opt  into  a smaller  group  which  consists 
of  students  with  similar  literacy  levels 
and  social/ group  interaction  skills. 

We  have  found  that  social  contact  and 
acceptance  of  one's  literacy  level  is 
one  of  the  first  steps  many  lower  level 
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EVEN  START’S  PHILOSOPHICAL  APPROACH 

The  philosophical  approach  to  education  at  Even  Start  rests  under  a Whole  Language 
umbrella.  We  define  Whole  Language  as  a belief  that  language  is  central  to  learning,  that  learning 
is  easiest  when  it  is  from  whole  to  part  (when  it  is  in  authentic  contexts),  and  when  it  is  functional 
to  the  learner.  Whole  Language  also  empowers  the  learner,  by  operating  on  a belief  that  learning 
is  both  personal  and  social  and  that  educational  settings  must  be  learning  communities.  This 
philosophy  Includes  an  acceptance  of  all  learners  and  the  languages,  cultures,  and  experiences 
they  bring  to  their  education. 

The  Whole  Language  approach  for  adults  is  developmental  and  is  driven  by  the  experiences  of 
the  adult  learner,  which  means  that  instruction  must  build  on  and  connect  to  an  individual’s  life 
and  language  experiences.  Oral  and  written  language  experiences  must  be  purposeful,  functional 
and  real.  The  four  language  modes  — listening,  speaking,  reading,  writing  — are  mutually  support- 
ive and  must  not  be  artificially  separated;  rather  they  should  be  equally  emphasized. 

The  Whole  Language  approach  also  invites  adults  to  recognize  and  build  upon  strengths.  Even 
Start  does  not  focus  on  deficiencies.  A broader  scope  of  assessment  and  instruction  is  incorpo- 
rated which  includes  assessing  the  student's  prior  knowledge,  providing  organizational  strengths 
and  options,  evaluating  new  learning,  and  identifying  and  teaching  concepts,  relationships  and 
connections. 

Students  are  empowered  when  they  become  active  agents  in  their  own  learning,  when  they 
acquire  a sense  of  belonging  paired  with  autonomy,  objectivity,  and  knowing  what  they  say 
matters.  Even  Start’s  curriculum  incorporates  strategies  to  encourage  self  growth  and  discovery 
of  individual  learning  styles  and  preferences. 


students  must  take  to  feel  comfortable 
with  themselves  and  be  motivated  to 
attend.  Additional  adult  literacy  experi- 
ences are  personalized  and  based  on  the 
goals  parents  set  for  themselves. 

Initial  Assessment  and  Measurement 
of  Adult  Learner  Progress 

Even  Start  is  an  empowerment  model, 
therefore  assessment  and  measurement 
of  adult  learners  is  not  based  on  their 
weaknesses;  rather,  it  is  based  upon  their 
strengths  and  the  wealth  of  knowledge 
they  bring  to  our  program.  Accordingly, 
our  assessment  tools  are  non-traditional 
and  seek  to  empower  and  strengthen  our 
parents/ adult  learners  from  the  moment 
they  come  into  the  program  by  revealing 
their  ability  to  take  charge  and  be  proud 
of  the  lives  they  are  leading. 


The  emphasis  at  Even  Start  is  on 
Family  Literacy  and  the  importance  of 
transferring  learning  from  parent  to 
child  and  visa  versa.  Therefore,  the  staff 
responsible  for  the  initial  assessment 
come  from  both  components  of  our 
program.  Parent/adult  learners  who 
enter  the  program  set  goals  with,  and 
are  initially  assessed  by,  both  a Family 
Literacy  Specialist  and  a Home  Based 
Visitor. 

During  the  intake  procedure,  the  staff 
informally  assess  the  functional /aca- 
demic capabilities  of  the  parent/adult 
learner  based  on  the  Amesbury  Even 
Start  Goal  Sheet  ( see  sample,  next  page) 
and  on  a variety  of  intake  sessions.  The 
Goal  Sheet  uncovers  a variety  of  assess- 
ment information,  but  at  the  same  time 
acts  as  an  extremely  inviting. 
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EVEN  START  IN  THE  HOME 

The  Home-Based  Visitor  for  each  family  ensures  the  family  receives  appropriate  services.  After 
the  full  team  identifies  goals,  the  Home-Based  Visitor  reviews  the  parenting  and  child  develop- 
ment goals,  and  assists  the  adults  in  designing  a program  to  fit  the  family’s  needs.  Information  and 
referrals  to  appropriate  community  services  may  be  offered  by  the  Home-Based  Visitors,  who 
can  act  as  advocates  for  adults  dealing  with  other  agencies.  Group  meetings  with  providers  from 
these  other  agencies  are  common,  and  are  critical  to  ensure  no  duplication  of  service  occurs. 

Visits  to  the  home,  usually  weekly,  incorporate  activities  related  to  various  goals.  During 
these  visits,  the  Home-Based  Visitor  brings  a book  and  often  a game  or  resource  from  one  of  the 
many  kits  developed  by  Even  Start  staff  The  Home-Based  Visitor  introduces  the  material  to  the 
parent(s)  and  child,  demonstrates  use  of  the  material  while  directly  interacting  with  the  child, 
models  adult  behaviors  necessary  to  facilitate  the  child’s  exploration  and  experimentation,  and 
encourages  appropriate  parent-child  interplay.  Visits  typically  last  one  hour  and  include  a brief 
evaluation  of  the  activity  and  a discussion  of  how  the  activity  can  be  replayed  throughout  the 
coming  week. 

On  weeks  when  families  participate  together,  such  as  in  playgroups  and  workshops,  there 
are  no  home  visits. 


unintimidating  way  to  assess  initial 
capabilities,  attributes  and  interests.  The 
Goal  Sheet  is  a visual  representation  of 
an  eight-week  cycle.  It  is  broken  up  into 
categories  representing  program  require- 
ments at  Even  Start,  including:  adult 
education,  parenting  skills,  child  devel- 
opment and  parent/child  together  time. 
These  categories  are  then  subdivided 
into  short-term  and  long-term  goals 
columns. 

Although  simple  in  description,  this 
Goal  Sheet  provides  the  case  managers 
with  a wealth  of  initial  assessment 
information  without  the  use  of  extensive 
formalized  testing  or  frequently  inaccu- 
rate initial  self-evaluations.  Informal 
skills  assessed  include  communication 
skills,  such  as  listening  to  the  description 
of  the  Even  Start  program  options,  and 
formulating  questions  about  the  adult 
learner's  individualized  Family  Literacy 
Plan.  Both  verbal  and  written  expression 
of  goals  are  used  in  the  initial  assessment. 

The  Goal  Sheet  also  provides  informa- 


tion on  the  critical  thinking  abilities  of 
the  parent/ adult  learner.  For  example, 
questions  posed  may  include:  What  is 
the  difference  between  short  and  long 
term  goals?  Are  the  goals  realistic? 

The  Goal  Sheet  also  provides  informa- 
tion on  decision-making  skills  such  as 
the  following:  what  program  options 
will  be  emphasized  during  the  initial 
eight  week  cycle?  how  do  eight-week 
program  options  lead  to  long  term  goals. 

The  Goal  Sheet  offers  some  elements 
of  affective  measurement.  Parents/Adult 
learners  often  express  dreams  and 
wishes,  combined  with  "felt"  or  "real" 
strengths  and  weaknesses.  This  informa- 
tion is  extremely  important  when  it 
comes  to  cycle  evaluations,  because  it 
provides  the  parent/ adult  learner  an 
opportunity  to  ethically  analyze  how 
they  perceive  themselves,  and  whether 
their  perceptions  have  helped  or  hin- 
dered their  progress. 

Parenting  and  child  development 
goals  are  also  assessed  during  the  initial 
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intake  procedure.  Information  is  col- 
lected formally  through  the  use  of  the 
Parent/Child  interaction  forms  which 
attempt  to  uncover  issues  which  may 
require  future  support  (see  sample  at  the 
end  of  this  section). 

Intake  Utilizes  Many  Formats 

The  intake  process  allows  a variety  of 
information  to  be  gathered  in  a number 
of  ways:  discussion  sessions,  informal 
goal-setting  sessions,  formalized  ques- 
tionnaires, and  a formalized  adult 
assessment  tool  — the  California  Adult 
Student  Assessment  System,  CASAS. 

All  Even  Start  Programs  nationwide 
utilize  CASAS,  a measure  of  functional 
reading  ability. 

[Editor' s Note:  CASAS  is  "a  comprehen- 
sive educational  assessment  system  designed 
to  measure  competency-based  curriculum  for 
all  levels  of  Adult  Basic  Education  and 
English  as  a Second  Language,  including  a 
pre-vocational  curriculum ,"  according  to  the 
CASAS  Overview.  According  to  its  develop- 
ers, it  measures  functional  basic  skills  in 
reading,  math,  and  listening  comprehension, 
and  utilizes  authentic  assessment  or  applied 


performance  measures  on  oral  proficiency, 
writing  skills,  pre-employment  and  work 
maturity  competence,  and  critical  thinking 
skills.] 

Competencies  include  the  ability  to 
read  a newspaper  advertisement,  to  use 
a telephone  directory,  or  to  read  the 
instructions  on  a bottle  of  medicine,  etc. 
Results  are  integrated  into  the  adult 
learner's  individualized  Family  Literacy 
Plan  (see  sample  at  end  of  this  section). 

The  initial  assessment  is  a key  compo- 
nent of  the  intake  procedure,  and  takes 
roughly  four  hours  to  complete.  Orienta- 
tion and  program  description  sessions 
run  approximately  one  hour.  The 
CASAS  can  take  anywhere  from  thirty  to 
sixty  minutes  to  complete.  (There  is  no 
time  limit.)  The  questionnaires  and  goal- 
setting sessions  usually  take  about  one 
hour  to  complete.  Goals  can  be  analyzed, 
changed  or  continued,  based  on  what 
the  adult  learner  wants  to  do.  The 
CASAS  is  administered  every  six  months; 
each  level  has  a pre-  and  post  test. 

At  the  end  of  each  cycle,  the  indi- 
vidual accounts  of  each  session  are 
attached  to  that  cycle's  goals  worksheet. 


Although  simple 
in  description, 
the  Goal  Sheet 
provides  the  case 
managers  with  a 
wealth  of  initial 
assessment 
information 
without  the  use 
of  extensive 
formalized  testing 
or  frequently 
inaccurate  initial 
self  evaluations. 


EARLY  CHILDHOOD  EDUCATION 

The  Even  Start  Center  provides  a comprehensive  program  for  young  children.  Children  spend 
time  in  the  Center  while  parents  are  engaged  In  adult  literacy  work,  or  when  goals  for  the  child 
can  best  be  addressed  there.  The  Center’s  program  brings  together  children  of  varying  ages  in  a 
safe,  accepting  environment  where  they  develop  and  play  together.  Through  developmental^ 
appropriate  experiences  and  activities,  children  are  encouraged  to  acquire  and  practice  cognitive, 
language,  socialization  and  motor  skills.  They  also  learn  to  respect  each  others’  challenges  and 
individuality. 

As  with  all  Even  Start  programming,  parents  are  instrumental  in  planning  and  implementing  the 
Early  Childhood  Program.  This  supports  the  parents’  goal  of  I)  becoming  more  aware  of  the 
needs  and  learning  styles  of  their  own  children,  2)  successfully  communicating  with  a teacher 
about  their  children,  and  3)  creating  a community  of  adults  who  are  caring  for  children. 

Even  Start  recently  hired  an  Early  Childhood  Specialist  to  plan  and  implement  the  Early 
Childhood  Program  and  to  supervise  the  Child  Care  Assistant.  The  program  includes  two 
morning  playgroups  per  week.  Families  sign  up  for  two  or  three  playgroups  each  eight  week  cycle. 
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AHESBURY  EVEN  START  EVALUATION  FORM 

Evaluation:  8-Week  Cycle  from to 

Parent  Name Date 

Family  Literacy  Staff Home-Based  Visitor 

During  the  cycle  just  finished: 

WHAT  DID  I LEARN  ABOUT  MYSELF  AS  AN  ADULT  LEARNER? 

Computers  aren’t  as  hard  as  I thought  they  would  be.  I’m  getting  the  hang  of  it. 

WHAT  DID  I LEARN  ABOUT  MYSELF  AS  A PARENT? 

Sometimes  my  child  needs  my  attention  and  sometimes  my  child  really  needs  time  alone. 
WHAT  DID  I LEARN  ABOUT  MY  CHILD? 

I feel  as  if  I better  understand  how  my  child  sees  the  world,  and  also  that  they  are  very  compli. 
cated! 

DID  YOU  REACH  YOUR  SHORT  TERM  GOALS?  Somewhat 

IF  YES,  WHAT  WAS  MOST  HELPFUL  IN  ACHIEVING  THESE  GOALS? 

The  workshop  I attended  was  helpful  In  working  toward  my  child’s  development  goals.  The 
Resume  Group  showed  me  how  to  sell  myself  on  paper  in  order  to  someday  get  a job. 

IF  NO,  WHAT  DO  YOU  THINK  STOOD  IN  YOUR  WAY? 

Sometimes  I missed  a Home-Based  Visitor  meeting  because  I had  other  appointments  that  I 
couldn’t  change  but  I do  think  I set  aside  time  to  play  with  my  child. 

WHAT  EVEN  START  ACTIVITIES  WERE  MOST  HELPFUL? 

The  parent  group  was  very  supportive  and  I feel  like  I really  learned  a bt  from  the  groups  that 
I attended. 

BASED  ON  WHAT  YOU  LEARNED,  WHERE  DO  YOU  WANT  TO  GO  FROM  HERE? 

I would  like  to  attend  more  workshops. 


The  Evaluation  Form  contains  basic 
information  about  what  went  on  in  each 
session:  the  goal  of  the  activity,  what 
materials  were  used,  who  led  the  ses- 
sion, and  an  opportunity  for  the  parent/ 
adult  learner  to  discuss  what  he/she 
learned  and  how  he/ she  may  transfer 
some  idea,  attitude  or  skill  to  his/her 
children.  This  allows  reflection  on  past 
goals  and  related  implementation. 
Changes  in  adult  learner  goals  are 
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presently  ascertained  by  evaluating  the 
last  cycle,  e.g.  were  the  goals  realistic? 
New  goals  are  acknowledged  on  the 
succeeding  8-weekcycle  Goal  Sheet. 

Measurement:  the  Portfolio, 
the  Goal  Sheet,  and  More 
Parents /adult  learners  choose  their 
own  Even  Start  programming;  when  a 
parent/ adult  learner  decides  their  goals, 
chooses  what  programming  will  support 
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that  goal,  and  evaluates  their  progress 
during  the  cycle,  they  are  more  likely  to 
be  taking  charge  of  their  own  learning. 

An  adult  learner  portfolio  is  created  at 
initial  placement.  The  folder  eventually 
includes  writing  samples,  math  work, 
and  other  content  area  activities.  The 
samples  are  compared  from  time  to  time, 
and  at  the  end  of  every  8-week  cycle,  for 
differences  in  quality.  Markers  are  being 
developed  to  measure  changes  in  atti- 
tudes toward  education  in  general. 

These  markers  will  help  track  behavioral 
changes  — such  as  reading  for  pleasure, 
or  writing  a note  to  a child's  teacher,  or 
choosing  one  shopping  center  over 
another  — from  applying  skills  learned 
in  the  program.  These  important  behav- 
ioral changes  are  often  lost  in  traditional 
assessment  and  evaluation. 

The  Goal  Sheet  at  Even  Start  is  a 
multi-level  tool.  Used  appropriately,  it 
empowers  the  learner  by  allowing 
expression  and  realization  of  very 
specific  short-term  and  long-term  goals. 
The  system  is  used  in  self-assessment  as 
an  adult  learner  builds  on  the  informa- 
tion he/ she  reflects  upon,  and  as  the 
work  is  continuously  defined  and  re- 
defined for  the  future.  Responsibility  for 
the  success  of  the  student7 s program  is 
squarely  placed  upon  the  adult  learner. 
There  is  also  ample  opportunity  for 
constructive  critical  feedback  about  our 
program  which  we  value  and  incorpo- 
rate into  our  work. 


To  strengthen  the  evaluation,  staff 
observations  and  participant  interviews 
at  the  end  of  each  8-week  program  cycle 
also  help  to  determine  participants' 
progress  in  meeting  their  literacy  goals. 
Two  times  per  month,  families  will  be 
asked  to  complete  a short  activity  evalu- 
ation to  determine  if  project  activities  are 
meeting  their  needs  and  are  relevant  to 
their  goals/ plans,  enjoyment,  issues 
critical  to  them,  and  their  relationship 
with  their  child  or  children. 

From  the  process  of  on-going  assess- 
ment by  parents  at  our  center  has  come 
the  stated  need  for  development  of  skills 
leading  toward  employment,  so  that 
parents  can  become,  or  can  continue  to 
be,  adequate  providers  for  their  chil- 
dren's physical  as  well  as  emotional 
needs. 

With  this  in  mind.  Even  Start  has 
initiated  the  development  of  a "Next 
Steps"  program  for  those  parents  who 
may  be  moving  on  to  a certification 
program,  vocational  training,  or  some 
other  form  of  continuing  education.  New 
legislation  for  all  Even  Start  programs 
requires  that  service  be  continued  for 
parents /adult  learners  even  if  their 
children  no  longer  fall  between  the  ages 
of  birth  to  seven  years.  We  have  envi- 
sioned that  when  this  does  occur,  many 
of  the  continuing  parents/adult  learners 
will  be  in  need  of  academic  and  counsel- 
ing support  to  help  them  make  the 
transition  into  their  next  steps.  0* 


Parents/adult 
learners  choose  their 
own  Even  Start 
programming; 
when  a parent / 
adult  learner 
decides  their  goals, 
chooses  what 
programming  will 
support  that  goal, 
and  evaluates  their 
progress  during 
the  cycle,  they  are 
more  likely  to  be 
taking  charge  of 
their  own  learning. 
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The  Informal  Reading  Inventory: 
Highlighting  Connections  & Capabilities 


by 


Eileen  Barry 


Workplace  Education 
Project 

Labor  Education  Center 
UMass  Dartmouth 


Returning  to  school  after  many  years 
is  a high  risk  and  frightening  step 
for  most  adult  learners.  When  they  make 
the  courageous  decision  to  enroll  in 
classes,  it  is  important  to  provide  initial 
experiences  which  are  positive  and 
validating.  While  the  learners  have  skills 
which  require  improvement,  they  also 
must  be  reminded  of  all  that  they  know 
and  are  capable  of  doing  already.  It  is 
also  important  that  they  see  they  are 
joining  a community  of  learners  who 
share  similar  experiences,  frustrations, 
and  accomplishments. 

For  these  reasons,  we  have  decided 
that  the  TABE  (Test  of  Adult  Basic 
Education)  assessment  test,  which 
evaluates  reading  and  math  skills,  is 
unsuitable  for  our  program. 

The  Workplace  Education  Project 
offers  six  GED,  Pre-GED,  and  ESL 
classes  in  five  different  sites  in  New 
Bedford  and  Fall  River.  Originally,  the 
classes  were  comprised  primarily  of 
Portuguese  immigrants  who  were 
employed  in  garment  and  small  manu- 
facturing industries.  With  the  changing 
economy,  however,  a growing  number 
are  now  unemployed. 

Previously,  when  students  enrolled  in 
the  Pre-GED  and  GED  classes,  we  asked 
them  to  complete  the  TABE  Survey  test. 
Noting  the  high  level  of  anxiety  of  new 
students,  however,  it  always  bothered  us 
that  we  welcomed  them  to  the  program 
with  such  an  intimidating  and  frustrat- 
ing tool. 


After  the  students  completed  the 
assessment,  it  was  our  turn  to  struggle 
with  it.  We  felt  frustrated  as  we  looked 
at  the  answers,  both  correct  and  incor- 
rect, because  we  had  no  way  of  knowing 
why  the  student  chose  a particular 
answer. 

After  scoring  the  test,  the  struggle 
would  continue.  We  would  confer  with 
other  teachers  in  the  program  and 
determine,  based  upon  the  TABE  score, 
whether  the  learner  was  better  suited  for 
the  Pre-GED  or  GED  class.  We  would 
make  a "good  guess"  as  to  which  class  to 
place  the  students  and  we  would  watch 
how  they  did  in  class  to  determine  if  the 
level  of  the  class  truly  met  their  needs. 

Often,  we  would  find  that  we  had 
misplaced  the  learners  and  would  move 
them  to  a different  class.  We  usually 
found  that  the  learners'  comprehension 
skills  were  much  better  than  the  TABE 
had  indicated.  When  they  were  in  a 
relaxed  setting,  reading  relevant  and 
interesting  material,  and  explaining  their 
interpretations  and  reactions,  it  became 
clear  to  us  and  to  them  that  they  pos- 
sessed many  more  skills  than  the  TABE 
acknowledged. 

It  became  apparent  to  us  that  it  would 
be  necessary  to  develop  assessment  tools 
which  better  reflect  the  strengths  and 
needs  of  the  learner.  We  decided  to 
create  an  Informal  Reading  Inventory, 
comprised  of  relevant,  interesting  read- 
ing selections  followed  by  questions 
which  enable  the  learners  to  realize  their 
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strengths  as  well  as  areas  which  require 
improvement.  A math  assessment  was 
designed  which  focuses  on  what  learners 
already  know  and  which  accommodates 
the  European  backgrounds  of  most  of 
the  students.  We  also  created  an  interest 
survey  to  identify  learners'  goals,  inter- 
ests, and  preferred  learning  styles.  While 
all  three  tools  are  important  for  assessing 
our  learners,  this  article  will  focus  on  the 
Informal  Reading  Inventory.  [Editor's 
note:  see  articles  by  Kelly  and  Whiton  in 
Volume  One  of  Adventures  in  Assessment 
for  more  information  on  Informal  Reading 
Inventories  and  Miscue  Analysis.] 

FOLLOWING  THE  LEARNERS'  LEAD 
We  determined  that  an  ideal  source  of 
reading  material  for  the  Informal  Read- 
ing Inventory  (ERI)  was  the  learners 
themselves.  Previous  participants  in  the 
program  had  written  and  published  an 
anthology  of  writing  entitled  Learning  to 
Fly.  These  stories  covered  a wide  range 
of  issues  such  as  families,  automobiles, 
immigration,  and  learning.  [The  stories 
are  reprinted  following  this  narrative.]  After 
using  the  Raygor  Readability  Formula 
(see  box  below)  to  determine  the  reading 
levels  of  the  stories,  it  became  clear  that 
in  addition  to  a wide  range  of  subject 


matter  the  stories  also  reflected  a wide 
range  of  reading  levels.  After  supple- 
menting the  anthology  with  a few  stories 
taken  from  Sharing  Our  Thoughts , the 
SABES  Southeast  Writing  Magazine,  we 
had  stories  with  reading  levels  ranging 
from  2.5  to  10.0.  (It  is  important  to  note 
that  we  used  the  Raygor  Readability 
Formula  to  confirm  the  estimates  that  we 
had  made  about  the  reading  level  of  the 
passages.  We  thought  using  a standard- 
ized measure  would  add  more  validity 
to  the  IRI,  but  continue  to  believe  that 
readability  formulas  should  be  used 
with  caution  because  they  fail  to  con- 
sider many  factors,  such  as  level  of 
interest  and  learner  background,  which 
have  a strong  influence  on  reading 
comprehension.) 

After  choosing  the  stories,  we  devel- 
oped questions  to  accompany  each  of 
eleven  passages.  Some  questions  require 
short  answers  while  others  ask  for  more 
extensive  writing.  In  addition  to  assess- 
ing learners'  reading  abilities,  we  then 
have  writing  samples  to  review.  This  is 
an  added  benefit,  for  we  did  not  get  an 
indication  of  the  learners'  writing  when 
we  used  the  TABE  test.  Since  writing  is 
a required  skill  for  the  GED  test  and  a 
focus  of  the  class,  the  sample  offers 
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THE  RAYGOR  READABILITY  FORMULA 

(Baldwin  and  Kaufman,  1 979) 

1 . Count  out  a 1 00-word  passage  from  the  selection  or  reading  you  have  chosen.  If  the 
reading  is  long,  It  may  be  more  accurate  to  choose  a passage  from  the  beginning  and  from 
the  end  and  to  take  the  average.  If  they  are  different,  choose  a third  passage  and  find  the 
averages.  For  most  low  readings,  though,  one  100-word  passage  should  be  enough. 

2.  Count  the  number  of  sentences  In  each  passage.  Count  a half  sentence  as  .5. 

3.  Count  the  number  of  words  in  each  passage  containing  six  or  more  letters. 

4.  Find  the  points  on  the  Raygor  graph. 


Awn^e  words  per  100 
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important  information.  Multiple  choice 
questions  accompany  some  of  the  pas- 
sages since  the  learners  will  eventually 
be  required  to  answer  this  type  of  ques- 
tion, but  they  are  able  to  choose  more 
than  one  answer  if  they  wish.  If  the 
student  answers  the  question  correctly, 
we  know  that  she  or  he  has  understood 
the  passage  and  is  familiar  with  the 
multiple-choice  format.  If  the  learner 
chooses  more  than  one  answer,  we  gain 
insight  into  his  or  her  thought  processes. 

The  learners  receive  the  eleven  stories, 
each  copied  on  a separate  piece  of  paper 
with  a set  of  questions.  We  explain  that 
the  stories  were  written  by  people  who 
were  once  in  the  same  position  as  they. 
The  new  students  are  always  excited  by 
this  and  become  more  interested  in 
reading  the  material.  We  explain  further 
that  they  should  decide  which  stories 
they  want  to  read.  They  are  encouraged 
to  read  as  many  as  possible,  but  are 
assured  that  if  a story  is  too  difficult  or 
of  no  interest  to  them,  they  should  not 
feel  required  to  read  it. 

Because  the  Pre-GED  and  GED  classes 
meet  simultaneously  and  since  we  do 
not  have  a counselor  present  to  adminis- 
ter the  assessment,  we  are  unable  to 
listen  to  the  students  read  the  passages 
and  to  conduct  a miscue  analysis.  The 
learners  work  independently  and  read 
the  passages  silently. 

When  we  review  the  assessment,  we 
do  not  use  an  official  scoring  procedure 
and  do  not  determine  a grade  level. 
Basically,  we  use  the  information  and 
our  knowledge  of  the  levels  at  which  the 
GED  and  Pre-GED  classes  are  working 
to  place  the  new  student  with  the  group 
that  seems  most  appropriate.  Since  we 
have  implemented  the  use  of  the  IRI,  we 


have  a 99%  successful  placement  rate! 
(On  one  occasion,  a student's  nerves 
interfered  with  his  ability  to  read  the 
stories  and  answer  the  questions.) 

When  we  discuss  our  assessment  tools 
with  teachers  in  other  programs,  they 
often  raise  the  issue  of  funders  requiring 
TABE  scores  or  results  from  other 
standardized  tests.  It  is  important  to 
note  that  DOE-funded  programs  are  not 
mandated  to  use  standardized  tests.  If 
programs  are  receiving  funds  from  other 
sources  which  do  impose  this  require- 
ment, we  feel  that  it  would  still  benefit 
the  learners  and  the  teachers  to  use 
alternative  assessment  tools  in  the  initial 
assessment  phase.  After  students  adjust 
to  being  in  class,  they  could  then  com- 
plete the  standardized  test.  While  the 
scores  from  the  standardized  measure 
would  satisfy  funders,  the  results  from 
the  alternative  assessment  can  be  used 
effectively  in  practice. 

We  have  found  that  learners  are  the 
best  judges  of  their  own  abilities.  They 
read  the  stories  that  are  written  at  their 
level  and  answer  the  questions  within 
their  capabilities.  This  provides  a posi- 
tive initial  experience  because  they 
determine  what  they  are  capable  of 
doing  rather  than  focusing  on  what  they 
don't  yet  know. 

It  is  also  an  inspiring  experience  as 
they  read  the  words  of  people  who  were 
once  in  a similar  situation  and  who  have 
now  reached  their  goals  or  made  great 
progress.  Tools  such  as  the  Informal 
Reading  Inventory  meet  our  goal  to 
create  positive  initial  experiences  for 
learners  so  they  will  begin  to  feel  more 
confident  about  their  decision  to  return 
to  school  and  about  their  ability  to  reach 
their  goals.  « 
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#1.  LEARNING  TO  (FLY)  SWIM 

Back  in  the  summer  vacations  of  1 986,  I 
used  to  go  everyday  to  the  rocky  beach  in  my 
little  town.  I used  to  go  there  with  some  of 
my  best  friends  that  could  go  with  me.  They 
are  Gil,  Miguel  and  Alberto.  Some  of  my 
other  friends  were  not  allowed  by  their 
mothers.  They  wouldn't  let  them  go.  In  the 
beach  we  would  lie  ten  minutes  in  the  sun, 
then  after,  rt  was  time  for  the  bath  in  the 
small  tide  pools.  It  was  like  that  every  day, 
even  Sundays. 

The  only  problem  at  those  days  was  I 
didn't  know  how  to  swim.  Mipuel  did.  so  he 

w 1 ~ 

was  going  to  teach  us,  me  and  Alberto.  At 
first  I went  from  rock  to  rock,  first  a few 
meters  in  shallow  water.  After  half  an  hour 
of  lessons  we  would  get  tired,  so  we  would 
explore  the  pools.  Sponges,  sea  urchins,  jelly 
fish,  sea  stars,  algae  and  coral  were  some  of 
the  creatures  we  used  to  look  for.  Although 
we  didn't  know  anything  about  them,  we 
knew  that  sea  urchins  and  jelly  fish  were  the 
ones  to  stay  away  from. 

Like  I was  telling...  we  used  to  just  go 
from  rock  to  rock.  With  some  patience  from 
Miguel,  a month  after  we  already  knew  our 
way  across  larger  pools.  After  that  Miguel 
said  that  we  could  swim  on  the  open  sea. 

At  that  time  I felt  a new  kind  of  freedom  in 
a new  world,  but  I also  thought,  “Be  careful, 
this  new  world  is  unknown,  after  learning  to 
(fly)  I don't  want  to  fall.” 

Joao  Ferreira 


BEST  COPY  AVA5LABLE 


#2.  MY  TRUCK 

It  was  one  of  the  last  ones  built  in  1 9S0,  by 
the  Chevrolet  Company.  It  is  a 1 950  Chevro- 
let, half-ton  pick  up  with  several  1 9S I parts.  A 
rugged  truck  built  for  work,  it  came  from  the 
factory  without  turn  signals,  seat  belts, 
carpeting  or  a radio. 

Bought  by  a contractor,  the  truck  was  put 
to  work  right  away.  Everyday  the  truck  was 
used  to  haul  tools  and  supplies  to  and  from 
different  jobs.  This  truck  was  worked  for  nine 
years  without  any  major  problems.  In  1 9S9 
the  contractor  gave  up  his  business  and  sold 
the  truck. 

A carpenter  who  had  worked  for  the 
contractor,  bought  the  truck  for  one  dollar. 

He  added  a few  accessories,  ladder  racks, 
turn  signals  and  three  wooden  tool  boxes. 
Now  the  truck  was  ready  to  go  back  to 
work.  For  ten  more  years  the  truck  was  used 
for  work.  The  motor  was  worn,  the  rear 
springs  were  shot  from  being  over-loaded. 
There  were  dents  and  rust  from  years  of 
work  and  bad  weather.  Still  running,  it  was 
taken  off  the  road;  replaced  by  a station 
wagon  in  1 969. 

I was  home  on  leave  when  I saw  it  in  the 
back  yard.  I looked  it  over  and  knew  it 
needed  a lot  of  work.  The  more  I thought 
about  it,  the  more  I wanted  it.  The  last  day  of 
my  leave  I bought  it.  It  was  put  in  a garage  for 
storage. 

It  was  late  in  1 974  when  the  truck  was  put 
back  on  the  road.  Now  there  was  a newer 
Pontiac  motor  and  transmission  under  the 
hood.  The  rear  axle  and  springs  had  also  been 
replaced.  The  electrical  system  also  had  to  be 
converted  from  six  volts  to  twelve  volts.  It 
needed  some  bodywork  but  that  would  wait. 

Put  together,  mainly  for  transportation,  it 
wasn't  long  before  the  truck  was  put  back  to 
work.  There  were  moving  jobs,  runs  to  the 
dump  and  scrap  yards.  It  also  did  a few  tow 
jobs  and  hauled  a trailer.  In  1 976  the  truck 
was  put  back  into  storage,  replaced  by  a van. 
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It's  now  1991,  after  I A years  of  storage 
the  truck  has  been  brought  back  to  life. 

This  time  it  has  been  completely  rewired  to 
accommodate  all  the  new  accessories.  Under 
the  hood  is  a new  350  cubic  inch  Chevy 
motor.  In  need  of  some  body  work  and  paint, 
it  sits  patiently  in  a heated  garage  waiting  for 
the  day  it  will  again  be  on  the  road. 

This  time  will  be  different.  Its  working 
days  are  over.  The  next  time  it  sees  the  road, 
it  will  be  for  cruising  and  transportation. 

Eddie  De  Mello  Jr. 


#3.  THE  FAMILY 

The  family  is  a place  where  people  bve 
and  feel  loved,  a place  of  understanding  and 
forgiving,  a place  of  happiness  and  sometimes 
sadness.  The  family  is  where  everybody  feels 
comforted.  It  is  where  sometimes  we  have 
discussions  to  fix  things,  a place  where 
everybody  respects  and  is  respected.  Family 
is  a place  where  we  form  a small  society  to 
go  in  a big  society.  I think  the  family  is  the 
most  important  thing  we  all  have  because  all 
of  us  in  the  family  are  loved  and  feel  loved. 
Everybody  helps  everybody. 

Good  things  happen  like  a marriage  or  a 
new  born  child  and  it  makes  us  happy. 
Sometimes,  when  somebody  gets  sick  or 
dies,  sadness  invades  the  family.  A family  is 
where  parents  respect  children  and  children 
respect  parents  and  grandparents,  where 
sometimes  discussions  arise  because  of  how 
children  behave,  because  of  money  or 
lifestyles.  If  we  are  good  families,  societies 
will  be  good  too.  In  conclusion,  family  is  the 
best  thing  everybody  has:  that  goes  from  bve 
to  all  other  good  things  that  can  exist.  I don’t 
know  what  I would  be  without  my  family. 

(I  feel  very  sorry  for  people  who  don’t  have  a 
family,  especially  the  homeless  or  people  who 
live  alone.) 

Maria  A.  Alves 


#4.  CAR  TECHNOLOGY 

One  of  the  most  competitive  sectors  of  the 
world’s  economy  is  the  automotive  sector.  So 
companies  worldwide  are  largely  tuned  into 
research  and  technological  development. 

They  don’t  only  need  technology  for  quality, 
but  also  to  be  a step  ahead  of  the  competi- 
tion. In  the  last  few  years,  the  most  amazing 
of  these  devebpments  are  active  suspension 
and  four  wheel  steering. 

In  the  case  of  active  suspensions,  they  are 
still  in  the  development  phase.  Some  firms  are 
taking  the  risk  to  sell  them  commercially. 
These  are  Mitsubishi,  Dodge,  and  Infiniti. 
There  are  also  the  “semi-active”  suspension 
which  is  standard  equipment  on  all  Lincoln 
models.  It  has  been  just  a couple  of  years 
since  Lotus  Formula  I car  used  active  suspen- 
sion. Lotus  is  still  one  of  the  technobgy 
leaders.  Active  suspension  is  one  of  the  most 
important  recent  automotive  devebpments, 
which  contributes  to  a bumpless  ride. 

Four  wheel  steering  has  been  a long  time 
dream  for  major  car  companies.  It  was 
brought  to  life  by  Honda,  using  a simple 
mechanic  system  within  its  model  “Prelude 
SI.”  Mazda  also  developed  a more  complicated 
electronic  system.  Both  proved  to  be  a big 
improvement  in  performance  and  are  also  a 
good  selling  point. 

The  list  could  go  on  with  aerodynamics, 
alternative  fuels,  and  “Kansei”  engineering. 

But  in  conclusion,  one  can  say  that,  “Car 
industry  is  the  first  to  work  for  consumers.” 
Now  we  can  be  sure  that  companies  are 
trying  to  do  the  best  for  us  and  our  environ- 
ment. 

Joao  Ferreira 
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#5.  MY  FAMILY 

We  were  four  brothers  in  the  family.  Until 
I came  to  the  USA,  I lived  in  a city  called 
Curitaba.  That’s  the  capital  of  the  Parama's 
state. 

In  1 983  we  lost  our  brother  and  my  house 
stayed  sad.  We  missed  him. 

In  1 984  we  adopted  a little  girl.  The 
happiness  came  again  to  us  because  she  is  a 
smart  girl. 

Now  we  feel  the  loss  of  my  brother,  but 
we  also  have  a little  girl. 

Samuel  Sales  Plres 
#6.  WAR 

As  a child,  growing  up  In  World  War  Two, 
I found  it  to  be  very  exciting.  Being  only  six 
years  old,  I thought  it  was  great  when  the 
sirens  went  off.  We  had  to  shut  all  the  lights 
off  and  pull  the  window  shades  down. 

My  sisters  and  I would  run  to  the  windows 
and  peek  out  at  the  air  raid  wardens.  They 
walked  up  and  down  the  streets  protecting 
our  neighborhoods.  We  were  unaware,  I 
know  at  our  ages,  of  the  Jewish  people  and 
how  they  were  murdered.  Also  innocent 
people  who  were  killed  trying  to  hide  them 
from  the  German  soldiers.  We  were  too 
young  to  read  a paper  or  to  understand  what 
was  being  said  on  the  radio. 

Today  I am  reliving  another  war.  This  time 
I am  fully  aware  of  what  is  going  on,  and  I do 
not  find  it  very  exciting.  I only  hope  the 
young  men  and  women  who  are  in  Saudi 
Arabia  know  what  they  are  fighting  for. 

It  is  sad  that  we  have  to  go  to  war  in  other 
countries  to  help  them  fight  for  their  free- 
dom. At  the  same  time  people  in  the  United 
States  take  their  freedom  for  granted.  We 
are  lucky  to  be  living  in  a democracy  where 
we  are  free  to  speak  out  for  what  we  believe 
in,  to  go  to  the  church  of  our  choice,  many  of 
us  are  never  satisfied.  We  should  look 
around  us  and  see  how  other  people  live,  and 
we  would  appreciate  what  we  have. 

Betty  Coderre 


#7.  HARBINGER 

Jon  could  still  see  the  mynah,  his  mind  still 
heard  its  piercing  cry.  Jon  had  seen  this  self- 
same four  previous  times  in  his  life.  Four 
dreams  that  he  could  never  forget,  and  now 
for  the  fifth  time  he  did  it.  Invaded  his 
dreams.  Another  nightmare  that  would  haunt 
him  throughout  his  days. 

He  was  ten  the  first  time  the  dream 
appeared  to  him.  He  wakes  up  screaming  and 
his  mother  comes  in  to  comfort  him.  She  tells 
him  it  was  only  a bad  dream.  The  next  day 
Jon’s  grandfather  died  of  a heart  attack.  It  was 
a bad  dream,  nothing  more. 

At  1 4,  the  dream  returned.  Once  again  Jon 
was  terrified  and  he  somehow  remembered 
having  seen  that  bird  some  other  time.  The 
next  day  there  was  a substitute  teacher  at 
school.  His  class  was  told  that  their  teacher 
had  passed  on.  No  further  explanation  was 
given  the  students.  Jon  knew  when  he  had 
seen  the  mynah  before,  and  he  was  scared. 

Once  more  did  Jon  dream  of  the  damned 
mynah.  He  was  1 9,  and  in  college  now  when 
he  was  called  in  to  identify  the  body  of  his 
roommate.  His  best  friend’s  cycle  had  been 
hit  by  a truck. 

The  next  nightmare  of  the  bird  is  the 
hardest  for  Jon  to  accept.  For  he  remembers 
it  with  shame.  His  wife  of  three  years  had  just 
given  birth  to  a boy.  It  was  his  first  child.  He 
was  so  scared  for  them,  for  he  knew  the 
meaning  of  his  dream.  The  shame  he  recalls  is 
of  the  relief  he  felt  when  his  mother  called 
him  and  his  father  had  just  died  in  his  sleep. 

And  now  after  six  years,  the  dream  again. 
Jon  lay  in  bed  trembling.  Someone  he  loved 
had  died,  or  was  about  to  die.  He  knew  there 
was  nothing  he  could  do.  After  having 
composed  himself  as  best  he  could,  Jon  got 
out  of  bed,  he  cleaned  up,  dressed,  and  went 
to  his  kitchen  to  have  breakfast.  He  was 
finishing  up  his  coffee  as  the  phone  rang. 

Michael  Woodmansee 
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#8.  THE  DAY  WHEN  I IMMIGRATED 
TO  AMERICA 

It  was  a happy  day  thinking  that  I was 
coming  to  America.  Every  person  talked 
about  it  as  big,  rich,  and  beautiful  country. 

I was  anxious  to  come  to  see  if  it  was  true! 
Even  so,  I was  very  sad  to  leave  my  little  and 
beautiful  country  too. 

In  my  country  I wasn’t  rich  and  I had  to 
work  hard.  I left  my  family,  friends,  animals, 
rivers,  and  springs,  the  Sundays  when  we 
could  rest  and  dress  up,  the  dances  where  we 
could  find  a boy  friend. 

When  at  the  airport,  waiting  to  leave  on 
the  plane,  I was  sad  and  happy  and  scared. 
When  in  the  plane  I was  anxious  to  get  to 
Boston  to  know  how  it  would  be.  When  still 
in  the  air  I could  see  a dark  city.  Right  away  I 
could  see  the  difference  from  Lisbon  to 
Boston,  Lisbon  so  bright  and  colorful  and 
Boston  so  dark. 

When  out  of  the  airport,  my  grandfather 
was  waiting  with  the  taxi.  I was  so  happy  to 
see  him  but  that  taxi  was  so  old  and  dirty. 
Soon  I started  to  look  and  see  that  what 
people  there  were  telling  me  wasn’t  true,  so 
many  old  things,  the  houses  of  wood,  the 
food  wasn’t  too  good.  Right  away  I started  to 
remember  my  house  and  my  country  where  I 
grew  until  I came. 

It  was  like  I had  nothing.  Thank  God  my 
parents  and  brothers  came  with  me.  I was  so 
sad  nothing  would  brighten  me,  in  part  too 
because  I had  left  my  boyfriend.  It  was  so 
different  from  what  I used  to  do.  Days  passed 
and  I went  to  work.  It  was  so  different  from 
what  I used  to  do.  Soon  I started  in  piece 
work.  It  made  me  so  afraid,  seeing  the  man 
with  the  clock  taking  the  time. 

Days,  months  and  years  passed.  I went  to 
school  for  a short  time  during  the  night.  It 
was  very  difficult,  but  slowly  I learned  a little 
bit,  how  to  speak  and  read  a little.  Twenty 
and  five  years  already  passed.  I miss  my 
country  Portugal,  still.  Today  I love  this 
country,  America.  I am  a happy  citizen  too.  I 
forgot  all  the  bad  things  and  changed  them 


for  the  good  it  had  to  offer  me. 

Marla  C.  Alves 

#9.  A TOY 

It  was  long,  long  ago.  I was  about  five  years 
old.  My  grandmother  was  sitting  in  front  of 
her  bedroom’s  window.  She  had  called  for 
me.  When  I got  there  I saw  her  with  her 
glasses  on,  holding  gently  a rag  doll.  She  had 
just  finished  it.  My  grandmother  was  there, 
smiling,  while  holding  the  most  beautiful  doll 
that  I had  ever  seen.  She  gave  me  the  doll, 
just  like  that.  It  was  not  Christmas,  or  any 
other  kind  of  commemoration,  and  it  was  my 
first  and  the  only  toy  I had  in  ail  my  child- 
hood. 

My  mother  wanted  to  sell  the  doll  to  make 
some  money,  but  my  grandmother  said  there 
could  not  be  money  enough  in  the  world  that 
paid  for  her  granddaughter's  happiness,  and 
she  reaffirmed  her  intention  that  she  had 
made  that  doll  with  love  to  be  given  to  her 
granddaughter  Zana.  The  doll  doesn’t  exist 
anymore,  but  my  grandmother’s  words  and 
the  doll  image  still  live  in  my  heart.  I can 
clearly  remember  it. 

The  doll  body  was  made  from  white  cloth 
that  my  grandmother  used  to  make  sheets 
and  pillow  cases.  Her  underwear,  little 
panties,  was  light  pink,  the  same  kind  of  cloth 
that  my  grandmother  used  to  make  our  own 
underwear.  Her  dress  was  hand-knitted  from 
lamb  wool  that  my  grandmother  had  dyed 
pink  previously.  Her  socks  were  hand 
crocheted  from  pink  cotton  thread  and  her 
shoes  were  made  from  lamb  fur.  Her  hair 
was  made  from  brown  wool  and  her  eye- 
lashes and  eyebrows  from  black  cotton 
thread  and  her  lips  from  rep.  It  was  the  most 
beautiful  rag  doll  I had  ever  seen  in  my  life.  I 
felt  fascinated,  overjoyed,  delighted  with  my 
doll;  but  my  attention  almost  failed  on  a 
particularity  — my  doll  was  wearing  socks 
and  shoes,  while  I was  barefoot 

Marla  Z.  Santos 
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#10.  GED 

I quit  school  when  I was  1 6 years  old 
because  of  financial  reasons.  I had  to  help  my 
mother  with  the  bills  and  my  two  younger 
brothers.  I had  many  different  jobs.  My  last 
job  lasted  for  22  years.  I would  like  to  have 
stayed  there  until  I retired  but  plant  closure 
took  that  dream. 

When  I was  younger,  I could  walk  across 
the  street  and  get  a job.  That's  how  easy  it 
was.  If  I didn't  like  that  one,  I could  just  as 
easy  go  somewhere  else  and  would  get  hired 
on  the  spot. 

Now  plants,  corporations,  and  companies 
want  high  school  diplomas  and  if  you  don't 
have  one,  most  good  paying  jobs  won't  even 

rnn«l^ar  w/Mir  tnnl!«.»Un  .L...  ... 
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even  tell  you  that’s  why  you  didn't  get  hired 
There  are  a few  places  left  that  don't  require 
a diploma,  but  they  are  getting  fewer  and 
farther  apart 

I was  a stitcher,  driller,  solderer,  tire 
builder,  supervisor  and  injection  molder  to 
name  a few.  I have  learned  a lot  up  to  this 
point  When  I think  about  my  life  and  past 
jobs,  there  have  been  a lot  of  things  accom- 
plished and  one  or  two  failures  along  the  way. 

Getting  a GED  is  not  as  easy  as  it  sounds, 
for  my  anyway.  Through  the  classes  I am 
learning  all  over  again  and  I am  determined 
to  get  my  diploma.. 

Marie  Lucas 


#11.  THE  WISH 

Arthur  had  always  been  fascinated  with 
the  past  As  a young  boy,  he  used  to  collect 
old  stamps  and  coins.  Strange  pieces  of  metal 
and  rock  he  would  scrounge  in  his  neighbor- 
hood were  precious  Indian  relics  or  stone  age 
tools  to  his  young  imagination. 

Now  he  was  in  college  studying  ancient 
history.  For  the  summer,  he  was  lucky 
enough  to  be  included  in  the  field  trip  to 
Israel  to  work  at  an  archaeological  dig. 


It  was  uncomfortably  hot  the  day  that 
Arthur,  carrying  his  shovel  and  camera, 
climbed  down  into  a newly  excavated 
dwelling.  He  welcomed  the  cool  air  in  the 
dim  interior.  After  working  carefully  in  the 
north  comer  for  an  hour,  he  was  scraping 
his  shovel  through  the  sand  when  he  heard  it 
clink  against  metal.  His  heartbeat  increasing 
with  excitement,  he  bent  down  to  examine 
his  find.  It  was  an  old  brass  lantern,  rather 
sooty  near  the  lip. 

Pulling  out  his  shirttail,  Arthur  started  to 
buff  the  grime  off  his  lantern.  Immediately, 
he  began  to  feel  light  headed  and  dizzy;  the 
room  grew  darker  and  an  unsettling  feeling 
of  spinning  passed  through  him.  He  found 

Cl  irrni  tr\riaA  kw  nrooni r k v^n/sr. 
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congealed  into  the  shape  of  a man.  “I  am  the 
genie  of  the  lamp,"  the  voice  boomed.  “You 
have  awoken  me  from  my  eternal  slumber; 
your  wish  is  my  command!" 

Arthur  almost  fainted  from  the  shock.  He 
sweated  and  trembled.  Finally  he  managed  to 
speak  in  a high  nervous  voice.  “I  wish  to  go 
back  in  time  and  witness  the  construction  of 
the  Great  Pyramid  at  Giza!" 

No  sooner  than  the  last  word  left  his  lips, 
he  found  himself  in  the  desert  He  looked 
down  and  saw  that  he  was  wearing  a long 
white  robe.  He  looked  up  and  saw  a half 
built  pyramid.  Nearby  were  men  also 
dressed  in  long  white  robes.  They  were 
straining  to  push  a huge  stone  block  along 
little  wooden  rollers.  Arthur  looked  desper- 
ately for  his  lamp,  but  it  was  nowhere  to  be 
found.  The  crack  of  a whip  behind  his  right 
ear  startled  him  into  awareness.  Approach- 
ing was  a heavily  armoured  Egyptian  soldier 
flailing  his  whip.  “Get  back  to  work,"  he 
commanded,  “or  you  will  be  thrown  to  the 
jackals!". 

Timur  Turkdogen 
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QUESTIONS 

Story  #1 . Learning  to  (Fly)  Swim 

Story  #4.  Car  Technology 

1 . The  author  of  the  story  probably  lived  in 

1 . According  to  the  author,  one  of  the  most 

a.  New  York  City 

important  advances  in  car  technology  is 

b.  Portugal 

a.  alternative  fuels 

c.  New  Bedford 

b.  electronic  systems 

c.  four  wheel  steering 

2.  Miguel  taught  the  author  to 

a.  fly 

2.  The  author  believes  that  the  car  industry 

b.  explore  pools 

spends  a great  deal  of  time  on  research  due 

c.  swim 

to 

3.  What  do  you  think  is  the  main  idea  of  this 

a.  consumer  complaints 

b.  competition  from  other  car 

story? 

manufacturers 

Story  #2.  My  Truck 

c.  governmental  regulations 
3.  The  author  says  that  car  companies  are 

1 . Which  alteration  or  alterations  came  first? 

trying  to  do  the  best  for  us  and  our 
environment.  Do  you  agree?  Why  or 

a.  ladder  racks 

why  not? 

b.  carpeting 

c.  turn  signals 

d.  both  a and  b 

Story  #5.  My  Family 

e.  both  a and  c 

2.  It  is  clear  from  the  story  that  the  author 

1 . The  author  of  the  story  was  sad  because 
a.  he  moved  to  the  USA 

a.  is  a carpenter 

b.  his  brother  was  test 

b.  has  just  bought  a truck 

c.  his  brother  died 

c is  proud  of  his  truck 

2.  What  made  the  author  happy? 

Story  #3.  The  Family 

Story  #6.  War 

1 . The  author  believes  that  good  families 

a.  don’t  have  arguments 

1 . The  job  of  the  air  raid  warden  was  to 

b.  are  important  for  society 

a.  protect  neighborhoods 

c.  live  together  in  one  house 

b.  fight  in  World  War  Two 

c.  turn  sirens  off 

2.  What  would  be  a good  title  for  this  story? 

3.  On  another  piece  of  paper,  write  your 

2.  As  a young  girl,  the  author  thought  war  was 
exciting  because 

own  description  of  “a  family.” 

a.  innocent  people  were  killed 

b.  she  thought  it  was  a game 

c.  she  could  listen  to  war  stories  on 
the  radio 
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3.  Some  examples  of  democracy  mentioned  in 

2.  How  did  the  author  feel  at  work? 

the  story  are 

a.  excited 

a.  freedom  of  religion 

b.  nervous 

b.  freedom  of  choice 

c fortunate 

c.  freedom  of  expression 

d.  both  a and  b 

3.  How  did  the  author  describe  the  U.S.? 

e.  both  a and  c 

4.  The  author  believes  that 

Story  #9.  A Toy 

a.  people  in  the  United  States  take 

their  freedom  for  granted 

1 . From  the  story  we  learn  that  the  little  girl 

b.  it  is  good  to  fight  for  the  freedom 

a.  was  very  poor 

of  other  countries 

b.  had  many  toys 

c.  life  was  better  when  she  was 

c.  sold  the  doll  to  make  some 

younger 

money  for  her  family 

Stnr*#  4*7 

* / * 

2.  The  doll  was  important  to  the  little  girl 
■ 

occause 

1 . A “mynah’*  is  a 

a.  the  doll  wore  shoes 

b.  It  was  a gift  of  love 

a.  nightmare 

c the  doll  was  worth  a lot  of  money 

b.  story  about  death 

c.  bird 

3.  Describe  the  doll  in  your  own  words. 

2.  Jon  was  ashamed  when  his  father  died 

because 

Story  #10.  GED 

a.  he  dreamed  that  his  father  died 

b.  he  was  glad  that  his  baby  did  not 

1 . Why  does  the  author  want  her  GED? 

die 

c.  he  had  an  argument  with  his  father 

2.  Why  do  you  want  your  GED? 

3.  What  do  you  think  happens  after  the 

phone  rings  at  the  end  of  the  story? 

Story  #1 1.  The  Wish 

(use  another  paper  if  you’d  like) 

Story  #8.  The  Day  When  1 Immigrated 

1 . What  is  the  theme  of  “The  Wish?” 

a.  Exploring  is  dangerous. 

b.  Be  careful  what  you  wish  for. 

to  America 

c.  It’s  better  to  live  in  the  present. 

1 . The  author  was  excited  to  come  to  the 

2.  Write  what  you  think  will  happen  to 

U.S.  because 

Arthur  next. 

a.  she  expected  a beautiful,  rich 
country 

b.  she  wanted  to  find  a job 

c.  she  didn’t  like  her  country 
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SAMPLE  ANSWERS  TO  STORIES  10  & I I 
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2.  Why  do  you  want  your  GED? 

X rtAArt  m.  G£t>  teJMlE.  TobpO  tbmPAtifc 
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Story  #11  The  Wish 


1.  What  is  the  theme  of  "The  Wish?" 

a.  Exploring  is  dangerous.. 

(by  Be  careful  what  you  wish  for. 
c.  It's  better  to  live  in  the  present 


2.  Write  what  you  think  will  happen  to  Arthur  next. 
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The  ESL  Classroom  as  Community; 
How  Self  Assessment  Can  Work 


by 

Dulany 

Alexander 


Operation  Bootstrap, 
Lynn 


1 teach  English  as  a second  language  at 
Operation  Bootstrap  in  Lynn.  Our 
population  is  mostly  Hispanic  adults. 
We  have  three  levels,  with  about  75 
students  per  year  enrolled.  One-third  of 
our  students  are  welfare  recipients,  one- 
third  are  funded  under  our  Department 
of  Education  grant,  and  one-third  are 
funded  under  JTPA  monies.  This  means 
that  our  funders  mandate  the  BEST  test 
for  measuring  both  student  gains  and 
our  program's  performance. 

This  definition  of  progress  as  "moving 
from  one  BEST  level  to  the  next"  can 
mean,  that  for  learners  who  don't  keep 
"progressing,"  we  don't  get  paid.  It  can 
also  leave  little  room  for  learners  or 
teachers  to  assess  learning  as  it  relates  to 
learner  goals  or  interests  or  to  how  the 
curriculum  or  instruction  meets  these 
goals  or  interests. 

It  is  for  all  these  reasons  and  more 
that  we  decided  to  focus  on  developing 
assessment  tools  that  would  help  us  and 
our  students  look  at  what  and  how  they 
were  learning. 

WHY  WE  TOOK  A NEW  LOOK 

Some  time  ago,  we  noted  a pattern  in 
our  student  enrollments  and  "termina- 
tions." Students  who  entered  the 
program  at  Level  1 were  likely  to  ad- 
vance to  Level  2,  yet  few  of  them 
advanced  to  Level  3.  Level  3 was  largely 
comprised  of  students  who  entered  at 
Level  3 or  who  had  spent  only  a brief 
time  in  Level  2. 


One  of  the  factors  at  work  was  dis- 
couragement. After  the  dramatic 
learning  curve  that  is  almost  certain  to 
result  from  attending  class  for  20  hours  a 
week,  the  normal  leveling  of  the  curve  at 
Level  2 felt  to  the  students  like  failure, 
either  theirs  or  ours.  Ours  is  a student 
centered  program;  the  students  can  see 
that  their  choices  determine  what  is 
taught  in  class.  Yet  we  were  failing  to 
help  students  gain  control  over  what 
and  how  they  learned. 

In  an  attempt  to  start  thinking  about 
the  what  and  how  of  learning,  I devised 
a weekly  evaluation  form  to  use  within 
our  program.  Some  of  my  goals  were: 

For  Me: 

>■  To  have  a better  sense  of  what's 
happening  in  the  students'  lives  so 
that  we  can  build  on  that  in  class 

>■  To  know  which  activities  engage 
them 

>■  To  address  short-term  problems 

>■  To  understand  each  student's 
measure  of  success. 

For  the  Students: 

>•  To  see  the  week  as  a collection  of 
activities 

>■  To  recognize  how  and  when  English 
is  used  outside  of  class  (and  how  to 
extend  those  activities) 

> To  separate  personality  (notably  the 
teacher's)  from  classwork,  to  be  able 
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to  critique  the  activities  without 
anyone  feeling  defensive 


>■ 


To  help  determine  the  direction  of 
the  class 


I 


>■ 


To  isolate  problems  so  they  become 
workable 


>■  To  identify  individual  growth  and 
successes 

>■  To  learn  to  set  (and  articulate)  short 
term  goals. 

We  used  the  weekly  evaluation  form 
(see  appendix)  in  our  program  at  all 


three  levels  for  several  months. 
Although  the  class  time  necessary  to 
complete  the  evaluation  decreased  as 
the  students  became  more  familiar  with 
it,  it  was  a pretty  dry  activity  that  quali- 
fied as  the  week's  most  boring  activity. 
Gradually,  each  teacher  found  other 
ways  to  meet  her  own  goals. 

GROUP  ASSESSMENT 
I came  to  see  that  my  goals  could  be 
better  served  by  other  means  than  this 
form.  For  example,  criticism  of  the  class 
activities  didn't  need  to  be  a weekly 


WEEKLY  EVALUATION  FORM 

Name:  # Hrs  In  Class:  Week  Ending: 

1 ) In  class  or  out: 

I learned  these  new  words: 

I practiced  this  grammar 

My  biggest  problem  was: 

My  biggest  success  was: 

2)  What/where/when/how  often: 

I read: 

I spoke: 

I wrote: 

I listened: 

3)  Class  activities  this  weelc 

The  most  fun: 

The  most  boring: 

The  most  helpfid: 

The  most  confusing: 

4)  I need  help  with: 
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Although  the  class 
time  necessary  to 
complete  the 
evaluation 
decreased  as  the 
students  became 
more  familiar  with 
it,  it  was  a pretty 
dry  activity  that 
qualified  as  the 
week's  most  boring 
activity.  Gradually, 
each  teacher  found 
other  ways  to  meet 
her  own  goals. 
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What  is 

indispensable  is 
that  the  classroom 
activities  include 
asking  each 
student  to  focus 
on  his  or  her 
specific  learning 
process:  what's 
working,  what's 
not  working,  how 
are  you  attacking 
the  problem,  and 
how  can  you  be 
helped? 


event  or  done  individually.  It  is  easily 
done  as  a group  activity.  We  now  peri- 
odically brainstorm  a list  of  the  week's 
activities  and  the  students  "vote"  with 
one  of  four  descriptors  that  the  group 
chooses  (fun,  helpful,  interesting,  confus- 
ing...). The  vote  invariably  shows  that 
every  activity  suits  some  students  and 
not  others. 

What  is  indispensable  is  that  the 
classroom  activities  include  asking  each 
student  to  focus  on  his  or  her  specific 
learning  process:  what's  working,  what's 
not  working,  how  are  you  attacking  the 
problem,  and  how  can  you  be  helped?  If 
the  student  has  no  clearer  idea  of  his  or 
her  own  situation  than  "I  need  help  with 
English,"  then  the  process  of  learning 
this  language  is  completely  out  of  his  or 
her  control.  If  it  doesn't  seem  to  be 
happening,  the  student  sees  no  choices 
other  than  to  give  up  or  try  to  find 
another  teacher  who  can  "make  it  hap- 
pen." 

Few  of  our  students  attended  college; 
the  majority  experienced  too  few  suc- 
cesses to  remain  in  high  school  until 
graduation.  Now  they  are  adults  with 
other  responsibilities.  The  biggest  suc- 
cess may  be  finding  a new  apartment; 
the  biggest  problem  may  have  been  a 
sick  child.  These  events  could  certainly 
overshadow  any  language  acquisition 
goals.  If  that  is  the  reality  of  the 
student's  life,  student  and  teacher  need 
to  realize  that  language  learning  can  not 
always  be  the  highest  priority.  Some 
weeks,  intentionally  exposing  oneself  to 
English  reading/writing/speaking/ 
listening  wherever  possible  is  all  that 
can  be  done. 


SELF-ASSESSMENT  IN 
THE  CLASSROOM 

But  in  a more  ideal  week,  the  student 
needs  to  know  how  to  advance  the 
overall  goal  of  "learning  English."  The 
process  starts  with  a self-assessment 
exercise.  Most  students  have  a pretty 
good  sense  of  how  they  compare  to 
their  classmates  at  a particular  skill.  For 
most  students,  it  follows  that  each  has 
strengths  and  each  has  weaknesses.  In 
my  class,  I begin  by  asking  each  student 
to  name  which  is  the  hardest  and  which 
is  the  easiest  for  him/her:  reading, 
writing,  listening,  or  speaking. 

We  work  from  this  crude  self-assess- 
ment to  refine  our  definition  of  the 
problem.  For  example,  if  Carmen  says 
that  speaking  is  the  hardest  for  her,  does 
she  mean  pronunciation,  or  grammar,  or 
vocabulary,  or  shyness?  If  the  former  is 
her  answer,  the  group  brainstorms  ways 
for  her  to  work  on  pronunciation  in 
school  and  at  home.  If  another  student's 
strength  is  pronunciation,  we  look  for 
ways  to  use  that  student  as  a resource. 
We  work  with  a volunteer  tutor,  if 
appropriate.  Having  agreed  on  what 
language  issue  is  the  special  target  of 
each  student,  we  as  a group  can  be  ready 
to  offer  ongoing  suggestions  and  encour- 
agement. More  important  in  the  long  run 
is  that  the  student  learns  to  use  self- 
assessment  and  goal  setting  as  tools  to 
manage  the  immense  task  of  learning 
English. 

Language  learning  is  a lengthy  pro- 
cess for  most  of  us.  None  of  our  students 
will  master  English  before  leaving  our 
program.  Some  students  will  leave  to 
work,  some  because  they  must  move 
too  far  from  school,  some  because  their 
families  can't  afford  their  time  for 
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school,  and  some  because  they've  de- 
cided that  "perfect"  English  is  not  a 
realistic  short-term  goal.  The  skills  that 
enable  the  student  to  treat  learning 


English  as  a personal  project,  whether  in 
school  or  out,  begin  with  assessment  and 
goal  setting.  These  are  probably  as 
valuable  as  anything  else  we  teach.  \0 
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Tape  Journals  in  the  Oral  Skills  Class 


by 

Eileen  Hughes 


LaGuardia 
Community  College 


This  entry  on  oral  skills  is  from  Literacy 
Update,  a publication  of  the  Literacy  Assis- 
tance Center.  It  is  reprinted  with  permission. 
For  more  information  about  the  Center  or 
Literacy  Update,  write  to  them  at  15  Dutch 
Street,  4th  floor,  New  York,  NY  10038,  or 
call  212-267-5309. 

In  an  oral  skills  or  pronunciation  class, 
students  can  be  assigned  a spoken 
journal  on  cassette  tape  just  as  they  are 
assigned  a written  journal  in  a writing 
class.  The  tape  journal  is  as  valuable  as 
the  written  in  promoting  fluency.  It 
becomes  a record  of  students'  individual 
explorations  in  English  and  provides 
the  medium  for  a dialogue  between  the 
teacher  and  each  student. 

PROCEDURE 

Ask  students  to  purchase  a cassette  tape. 
Tell  them  they  are  responsible  for  com- 
pleting a fixed  number  of  "entries" 
during  the  course.  Ask  them  to  speak  on 
a given  topic  for  at  least  five  minutes 
each  week  and  explain  that,  after  they 
hand  in  the  journal,  the  teacher  will 
listen  to  it  and  respond  on  the  tape.  You 
can  suggest  that  they  not  read  (from 
printed  material),  but  instead  speak 
spontaneously,  stopping  the  tape  where 
they  need  to  and  continuing  again. 

When  you  return  the  tapes,  instruct 
students  to  listen  to  their  own  voice  and 
then  to  the  teacher's  comments.  Next, 
they  should  bring  the  tape  back  to  the 
end  of  their  last  recording  and  make  a 


new  entry,  taping  over  the  teacher's 
comments  (which  may  be  long-winded!). 

If  possible,  give  students  access  to  a 
tape  recorder  before  class  begins  or 
during  break-time,  in  case  they  do  not 
have  a recorder  at  home. 

Try  to  return  the  tapes  with  your 
comments  to  the  students  as  soon  as 
possible.  (It's  a good  idea  to  have  the 
students  hand  in  their  tapes  on  different 
days.  This  lightens  the  teacher's  burden.) 

ASSIGNMENTS 
The  first  assignment  should  allow 
students  to  relax  and  get  comfortable 
with  the  medium.  Suggestions:  tell  me 
about  yourself. . .your  family.. . .your 
country.  Further  suggestions  include: 
What  makes  you  laugh?  Tell  me  about 
something  you  are  good  at  doing.. .about 
a good  friend... about  a dream  you  had. 

Give  assignments  focused  on  improv- 
ing students'  weakest  skills.  For 
example,  for  practice  pronouncing  past 
tense  endings:  Tell  me  about  an  experi- 
ence in  the  past,  either  something  that 
happened  in  your  country  or  something 
that  happened  during  your  first  days  in 
New  York. 

From  here,  assignments  can  become 
individualized;  each  entry  can  set  the 
direction  that  the  next  one  will  take. 

FEEDBACK 

Answer  genuinely.  Thank  the  stu- 
dents for  sharing  their  stories,  adven- 
tures, and  often  remarkable  insights. 
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As  with  a written  journal,  it  can  be 
argued  that  teachers'  comments  should 
focus  on  content  and  not  point  out 
errors,  although  the  latter  is  sometimes 
irresistible! 

Note:  It's  a good  idea  to  make  notes  as 
you  listen  to  the  students'  tapes.  This 
will  improve  your  memory  when  re- 
sponding and  will  also  become  a useful 
record  of  students'  needs,  interests  and 
history. 

SELF-ASSESSMENT 

Ask  students,  once  they've  become 
comfortable  with  the  medium:  Tell  me 
how  you  feel  in  the  class.  What  do  you 
like  best  about  the  class?  Least?  How 
can  the  class  help  you  more?  (Be  sure  to 
provide  examples:  more  listening  exer- 
cises? more  homework?  less  homework? 
more  pronunciation?)  Or:  Listen  to  your 
own  voice  on  the  tape.  What  words  are 
hardest  to  understand?  What  are  your 
strongest  skills?  your  weakest  skills? 
(Again,  provide  examples.) 

Elicit  from  your  students  (periodically 
or  midway  through  the  course)  what 


they  feel  is  most  valuable  about  this 
experience  and  discuss  the  tapes  in  class. 
Remind  students  upon  completion  of  the 
course  of  the  value  of  the  tape  journal. 
Encourage  them  to  continue  with  it,  as 
they  would  a written  journal,  even  after 
the  class  ends. 

USES 

>•  For  diagnostic  purposes:  common 
and  consistent  errors  can  become  the 
focus  of  future  lessons. 

>•  For  assessment  of  fluency  of  speech 
and  contextualized  pronunciation. 

>•  For  building  students'  awareness  of 
their  own  strengths  and  weaknesses. 

FURTHER  USES 

>•  For  class  evaluation:  students  often 
find  it  easier  to  make  discrete  and 
constructive  comments  on  the  tapes 
than  in  person. 

>■  For  building  students'  self-confi- 
dence: most  students  respond 
positively  to  hearing  themselves 
speaking  English,  [g' 


As  with  a written 
journal,  it  can 
be  argued  that 
teachers' 

comments  should 
focus  on  content 
and  not  point  out 
errors,  although 
the  latter  is 
sometimes 
irresistible! 
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What  Counts? 

Knowing  Math  and  Passing  the  GED 


by 

Sally  Spencer 


Care  Center,  Holyoke 


I have  been  the  math  teacher  at  the 
Care  Center,  a school  for  pregnant  and 
parenting  teenage  mothers  in  Holyoke, 
for  a little  over  a year.  This  by  no  means 
makes  me  an  authority  on  evaluation  in 
the  adult  mathematics  classroom.  But  as 
a member  of  the  ABE  Math  Team,  I've 
had  the  opportunity  to  think  about  it  a 
lot  lately.  I'd  like  to  share  some  ideas 
and  some  questions. 

In  November,  the  ABE  Math  Team 
received  a grant  from  the  National 
Institute  for  Literacy  through  Holyoke 
Community  College.  The  purpose  of  this 
grant  is  to  review  and  adapt  the  National 
Council  of  Teachers  of  Mathematics' 
Curriculum  and  Evaluation  Standards  for 
School  Mathematics  (the  Standards ) for  the 
adult  education  community.  To  accom- 
plish this,  the  ABE  Math  Team  has 
sub-divided  into  workgroups  focusing 
on  either  the  ESL,  the  ABE,  the  GED 
classroom  or  workplace  education.  It  is 
through  participating  in  the  ABE  Math 
Team  that  I have  been  focusing  my 
attention  on  the  "Evaluation  Standards" 
at  the  GED  level. 

The  Standards  were  written  as  a 
response  to  the  publication  in  1983  of 
A Nation  at  Risk  which  made  it  clear  that 
the  mathematics  education  which  the 
nation's  students  were  receiving  at  that 
time  was  inadequate.  Over  three  years, 
mathematics  educators  from  across  the 
nation  wrote,  tested,  revised  and,  in  1989, 
finally  adopted  the  Curriculum  and 
Evaluation  Standards  for  School  Mathematics. 


The  Standards  reflect  a new  vision 
of  what  it  means  to  "do"  and  "know" 
math.  In  its  introduction,  the  Standards 
lists  its  societal  goals  as:  mathematically 
literate  workers;  a citizenry  of  lifelong 
learners;  opportunity  for  all  learners; 
and  an  informed  electorate.  For  the 
individual  student,  that  means  it  is  no 
longer  enough  to  be  arithmetically 
proficient.  Instead,  the  Standards  hope  to 
ensure  that  students  will:  learn  to  value 
mathematics;  become  confident  in  their 
own  abilities;  become  mathematical 
problem  solvers;  and  be  able  to  commu- 
nicate and  reason  mathematically.  To 
that  end,  the  Standards  have  repriori- 
tized the  emphasis  on  various  topics  in 
math  instruction,  and  stress  new  meth- 
ods of  teaching  and  evaluation.  The 
curriculum  is  separated  into  three 
sections:  grades  K-4, 5-8  and  9-12;  evalu- 
ation is  the  final  fourth  part. 

So  how  can  this  national  mathematical 
movement  be  incorporated  in  the  ABE 
classroom?  How  does  that  impact 
assessment?  What  does  this  mean  for  the 
adult  learner? 

Some  of  the  issues  which  I have  been 
able  to  identify  which  make  evaluation 
in  the  GED  classroom  distinct  from  the 
traditional  K-12  setting  are: 

>•  grading  is  not  an  issue  in  the  adult 
classroom; 

> intake  evaluation  is  an  essential 
component  for  admission  to  the 
GED  classroom; 
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"The  vision  of  mathematics  education  in  the  standards  places  new  demands  on  instruction  and 
forces  us  to  reassess  the  manner  and  methods  by  which  we  chart  our  student's  progress.  In  an 
instructional  environment  that  demands  deeper  understanding  of  mathematics,  testing  instru- 
ments that  call  for  only  the  identification  of  single  correct  responses  no  longer  suffice.  Instead, 
our  instruments  must  reflect  the  scope  and  intent  of  our  instructional  program  to  have  students 
solve  problems,  reason  and  communicate.  Furthermore,  the  instruments  must  enable  the  teacher 
to  understand  student's  perceptions  of  mathematical  ideas  and  processes  and  their  ability  to 
function  in  a mathematical  context.  At  the  same  time,  they  must  be  sensitive  enough  to  help 
teachers  identify  individual  areas  of  difficulty  in  order  to  improve  instruction."  (NCTM  Standards ). 


many  adult  students  bring  to  the 
classroom  a fairly  traditional,  narrow 
and  rigid  preconception  of  what  it 
means  to  "know"  math,  and  a long 
history  and  a strong  sense  of  their 
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the  context  of  that  definition;  and, 
most  importantly. 


>•  the  ultimate  evaluation  for  the  GED 
student  is  successful  completion  of 


the  state  administered  GED  exam. 

How  do  these  differences  affect  the 
evaluation  standards?  I confess,  I am  one 
of  the  converted.  I believe  strongly  in  the 

vision  of  the  math  class  as  advocated  bv 

✓ 

the  NCTM.  As  part  of  my  work  with  the 
ABE  Math  Team,  I have  adapted  from 
page  191  of  the  Standards  the  following 
NCTM  evaluation  guidelines  for  the 
GED  classroom: 


INCREASED  ATTENTION 

Assessing  what  students  know  and  how 
they  think  about  mathematics 

Having  assessment  be  an  integral  part 

of  teaching 

Focusing  on  a broad  range  of 
mathematical  tasks  and  taking 
a holistic  view  of  mathematics 

Developing  problem  situations  that 
require  the  applications  of  a number 
of  mathematical  ideas 

Using  multiple  assessment  techniques, 
including  written,  oral  and  demonstra- 
tion formats  in  individual  and  In  group 

contexts 

Using  calculators,  computers  and 
manipulatives  in  some  assessment 
processes 

Using  the  standardized  GED  test  as* 
only  one  of  many  indicators  of  progress 


DECREASED  ATTENTION 

Assessing  what  students  do  not  know 


Having  assessment  be  simply  counting 
correct  answers 

Focusing  on  a large  number  of  specific 
and  isolated  skills 

Using  exercises  or  word  problems 
requiring  only  one  or  two  skills 


Using  only  individualized  written 
exercises  and  tests,  because  group 
work  is  cheating 


Excluding  calculators,  computers,  and 
manipulatives  from  all  assessment 
processes 

Using  the  standardized  GED  test  as  the 
only  indicator  of  success 
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WHAT 

COUNTS 


What  kinds  of 
assessment 
instruments, 
both  formal  and 
informal,  can  I 
find  or  develop  to 
help  bridge  the 
gap  between 
promoting, 
reasoning, 
communicating 
and  making 
connections  and 
" getting-the- 
right-answer"  on 
the  GED  test? 


Admittedly,  while  these  are  worthwhile 
goals  to  promote  students'  growth  as 
informed  citizens  and  empowered 
mathematicians,  it  is  equally  essential 
that  teachers  do  not  ignore  their  GED 
student's  priorities.  Passing  the  GED 
test,  which  still  tests  for  "single  correct 
responses",  is  the  primary  goal  of  the 
GED  student.  The  goals  of  developing  a 
problem  solving  attitude,  being  able  to 
communicate  in  mathematical  terms, 
being  able  to  reason  mathematically  and 
being  able  to  see  mathematical  connec- 
tions to  their  everyday  life  can  seem,  at 
best,  secondary  to  the  students  and,  at 
worst,  irrelevant  to  them. 

I believe  it  is  important  to  allow  time 
in  the  GED  mathematics  class  to  discuss 
these  additional  goals  so  that  the  stu- 
dents have  a chance  to  voice  their 
expectations  and  identify  and  incorpo- 


rate these  new  ideas  of  what  it  means  to 
"know"  math,  while  at  the  same  time 
acknowledging  that  passing  the  GED 
test  is  a long-term  objective. 

I am  planning  to  continue  to  explore 
ways  to  resolve  this  dichotomy.  My  next 
question  in  this  quest  is:  what  kinds  of 
assessment  instruments,  both  formal  and 
informal,  can  I find  or  develop  to  help 
bridge  the  gap  between  promoting, 
reasoning,  communicating  and  making 
connections  and  "getting-the-right- 
answer"  on  the  GED  test?  10 

Editor's  note:  Please  write  to  Sally  c/o 
Adventures  in  Assessment  your  thoughts, 
ideas,  or  assessment  instruments  that  explore 
this  dichotomy  in  teaching  math  in  a GED 
setting.  If  you  have  some  experience  with 
math  assessment  not  at  the  GED  level,  we 
welcome  hearing  from  you  as  well. 


O rURES  IN  ASSESSMENT 

ERIC 


Volume  4:  Spring  1993 


Voices  from  the  Field 

Through  the  Eyes 
of  an  ABE  Interviewer 


The  GOALS  Project  is  a field  test  of  10 
varied  components  designed  to  lead  to  the 
development  of  an  accountability  system 
for  adult  basic  education  programs  in  the 
Commonwealth  of  Massachusetts.  The 
project  was  initiated  by  the  Bureau  of  Adult 
Education  of  the  state's  Department  of 
Education,  and  has  stated  its  purpose  is  to 
"define  a set  of  indicators  of  program  quality 
which  are  educationally  sound,  useful  to 
students,  teachers  and  funders,  and  not 
excessively  time-consuming." 

Volume  3 of  Adventures  in  Assessment 
included  a dialogue  between  Editor  Loren 
McGrail  and  Lindy  Whiton,  who  coordi- 
nated that  part  of  the  project  which  focused 
on  alternative  assessment.  This  section  offers 
more  insight  into  the  GOALS  effort. 

Component  Seven  of  the  GOALS 
project  seeks  "to  determine  what  factors 
adult  learners  themselves  identify  in 
evaluating  their  goal  attainment."  My 
objective,  as  a Bureau  of  Adult  Educa- 
tion-trained ABE  interviewer,  was  to 
find  this  information  by  asking  five 
questions  to  students  across  the  state: 
What  brought  you  here  that  makes  you 
stay?  what  would  make  you  leave?  if 
you  were  in  charge  of  the  money,  where 
would  you  spend  more?,  and  what  are 
your  next  steps? 

With  these  questions  in  hand  and  my 
note-taking  skills  primed,  my  first  set  of 
interviews  took  place  in  early  Novem- 
ber, 1992  in  Dorchester  and  my  last  in 
early  January  in  Pittsfield.  [Editor's  note: 
The  project  interviewed  125  ABE  and  125 


ESL  students  overall  statewide.] 

I completed  more  than  twenty  inter- 
views with  students  from  Dorchester, 
Ware,  Orange,  Springfield,  Pittsfield  and 
Amesbury.  Some  interviewees  were 
nervous  because  they  did  not  know 
what  to  expect  but  as  word  got  out,  more 
students  wanted  to  be  included.  It  was 
true  for  me,  too;  the  more  interviews  I 
conducted,  the  more  at  ease  with  the 
process  I became. 

Before  each  interview  I used  the  first 
few  minutes  to  establish  a comfortable 
level.  I would  ask  about  children  and 
note  how  I had  never  visited  their 
program  or  area  of  the  state  before.  I 
would  then  read  all  five  questions  and 
ask  where  they  would  like  to  begin. 

Most  often,  we  would  start  with  the  first 
question  and  proceed  in  order.  Some 
interviews  lasted  sixty  minutes  while 
others  were  less  than  twenty. 

My  first  interviews  began  in  a home 
less  shelter  for  women  and  their 
children.  I interviewed  nine  women 
over  a threeday  period.  When  the  first 
student  came  in,  I introduced  myself, 
told  her  that  the  Department  of  Educa- 
tion was  using  these  interviews  for  a 
report  to  the  federal  government.  I told 
her  that,  since  we  consider  students  to  be 
the  experts,  she  was  making  history.  I 
reassured  her  that  the  interview  was 
fully  confidential.  I told  her  I would  be 
asking  five  questions  and  that  there  were 
no  right  or  wrong  answers.  This  became 
my  way  of  welcoming  interviewees. 


by 

Nancy  Jane 
Venator 


VISTA  Volunteer 
MA  Dept . of  Education 
Bureau  of  Adult 
Education 
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Having  a strong 
follow-up 
program  kept  her 
involved  and 
updated  about 
educational 
opportunities. 
Being  able  to  see 
her  child  at  lunch 
made  taking 
classes  more 
convenient  and 
stress-free.  She 
liked  to  learn  and 
wanted  to  be 
"somebody.'' 


I tried  to  prevent  my  note-taking  from 
being  distracting  by  explaining  it  and 
then  moving  on  with  the  questions. 

The  first  interview  went  well  and  took 
the  full  sixty  minutes.  I gathered  three 
pages  of  notes  and  lots  of  inspiration. 

Her  responses  were  nearly  typical  of 
future  interviews  with  mothers  but  what 
made  her  interview  so  special  was  how 
she  expressed  herself.  She  told  me  how 
having  her  child  say  to  her  in  the  morn- 
ing, "We  gotta  go  to  school,  get  up!"  on 
mornings  she  didn't  want  to  go,  encour- 
aged her  to  earn  her  GED.  Having  a 
strong  follow-up  program  kept  her 
involved  and  updated  about  educational 
opportunities.  Being  able  to  see  her  child 
at  lunch  made  taking  classes  more 
convenient  and  stress-free.  She  liked  to 
learn  and  wanted  to  be  "somebody."  She 
set  attainable  goals:  to  attend  college  and 
become  a nurse  because  she  liked  to  help 
people. 

My  most  memorable  interview  was 
with  a student  whose  writing  had  been 
selected  for  the  "Writing  On  the  Walls" 
project  that  I coordinate.  ("WOW"  is  a 
framed  collection  of  learner-generated 
writings  from  ABE  and  ESL  classes  from 
across  the  state.  The  collection  is  dis- 
played along  the  walls  leading  into  the 
Bureau  of  Adult  Education.)  I was  going 
to  wait  until  the  end  of  the  interview  to 
tell  him  I knew  of  his  writings  but  in  the 
middle  of  my  introduction  about  him 
making  history,  he  enthusiastically 
interjected,"  I already  did.  My  writing 
was  selected  for  Writing  On  the  Walls!" 

I was  so  moved  and  surprised  that  I 
revealed  my  involvement  with  WOW. 
We  talked  a bit  about  his  writing  and 
then  moved  onto  the  questions. 

Making  that  personal  connection 


made  a difference  since  it  established  a 
comfortable  conversation  level.  He  then 
eloquently  and  dramatically  described 
how  he  felt  when  he  could  not  read, 
"When  I could  not  read  and  went  into 
stores  there  was  an  animal  inside  of  me 
[he  touched  his  stomach].  As  I learned 
more  and  more  the  animal  slowly  went 
away  [he  moved  his  hand  slowly  up  his 
chest]  and  now  it  is  gone  forever  [he 
moved  his  hand  upward  and  out  with  a 
flourish]."  I will  never  forget  the  image. 
When  the  interview  finished  he  proudly 
read  to  me  and  showed  me  around  his 
learning  center. 

The  interview  with  a "WOW"  author 
confirmed  my  assumptions  about  how 
to  boost  students'  self-confidence,  yet,  I 
had  had  no  idea  how  much  having  one's 
writing  selected  and  displayed  could 
increase  one's  self-confidence. 

SOME  CONSISTENT  ANSWERS 

There  were  some  common  answers 
to  the  five  questions.  For  the  question, 
"What  brought  you  here?"  many  cited 
having  children  as  motivating  them  to 
return  to  school  so  they  could  provide 
better  for  their  families.  The  answer  to, 
"What  makes  you  stay?"  is  childcare.  At 
centers  with  on-site  childcare,  parents 
talked  of  how  the  child  could  play  with 
educational  games  and  make  friends 
while  they  studied.  Students  also  spoke 
of  the  caring  and  supportive  environ- 
ment provided  by  the  teachers.  While 
childcare  was  especially  helpful,  it  was 
clear  that  it  was  the  teachers  who  kept 
the  students  coming  everyday. 

When  asked,  "What  would  make  you 
leave?"  the  students  would  immediately 
respond,  "Nothing,"  and  then  wonder 
if  the  school  was  going  to  be  closed.  I 
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needed  to  assure  them  that  I was  not 
here  to  close  the  school.  After  they 
accepted  my  answer,  they  would  most 
often  respond  that  a job  or  getting  their 
GED  would  make  them  leave. 

The  most  challenging  and  difficult 
question  was,  "If  you  were  in  charge  of 
the  money,  where  would  you  spend 
more?"  They  either  responded  "I  don't 
know,"  and  then  started  talking  about 
books,  computers,  better/larger  facilities 
and  childcare,  or  they  would  approach 
the  question  from  a larger  perspective 
and  talk  about  increasing  binding 
nationwide  in  the  areas  of  education, 
childcare  and  homelessness.  Few  stu- 
dents answered  the  question  from  both 
points  of  view. 

The  students  were  fairly  quick  to 
answer  "What  are  your  next  steps?"  as 
if  they  had  been  seriously  thinking  about 
it  before  I asked.  Most  of  the  students 
planned  to  continue  their  education  by 
going  beyond  their  GEDs,  usually  at  a 
community  college.  The  students  would 
often  speak  of  their  future  careers  and 
how  these  fit  in  with  their  education 
plans. 

I re-read  the  Component  Seven  objec- 


tive, "to  determine  what  factors  adult 
learners  themselves  identify  in  evaluat- 
ing their  goal  attainment,"  after  all  my 
interviews  were  completed  and  it  was 
time  to  write  this  article.  I realized  that 
the  answers  to  "what  makes  you  stay?" 
and  "what  are  your  next  steps?"  were 
most  revealing  because  they  reveal  not 
only  personal  motivations  but  also  that 
the  student  is  conscious  of  her/his  best 
learning  environment.  As  one  student 
said,  "I  want  to  learn.  I like  the  teachers 
and  students.  I want  to  go  as  far  as  I can 
go...  friends  notice  the  difference." 
Learning  centers  that  combined  a strong 
commitment  to  the  individual  student's 
learning,  a pressure-free  environment, 
and  challenging  material  kept  students 
returning  daily  to  classes. 

Asking  about  next  steps  allowed  the 
students  to  discuss  their  ambitions  and 
goals.  A person's  realistic  individual 
goals  directly  reflects  the  extent  to  which 
the  student  has  been  encouraged  by  her 
learning  environment  to  recognize  her/ 
his  own  achievable  potential.  One 
student  said  it  best,  "I'm  facing  forward 
and  taking  the  biggest  step  I can."  gTf 


"I  want  to  learn. 

I like  the  teachers 
and  students. 

I want  to  go  as  far 
as  I can  go... 
friends  notice  the 
difference." 
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Publication  Review 

This  is  not  a lest 


This  is  not  a test  Dehli,  K.,  Dobbs,  G., 
Dwyer,  P.,  Hilton,  D.,  Horsman, }.  & 
Ingram,  L.  (1990).  Toronto:  East  End 
Literacy  Press. 


Review  by 
Don  Robishaw 


World  Education / 

SABES 

Boston 


This  kit  for  new  readers  contains 
many  useful  tools  and  exercises  for 
teachers  interested  in  implementing 
alternative  assessment  strategies  in  their 
respective  Adult  Literacy  programs.  It  is 
also  a good  introduction  to  the  novice  of 
the  various  options  to  testing  now  being 
developed  in  the  field  by  teachers. 

On  the  other  hand,  the  kit  can  also  be 
used  by  the  expert  teacher  who  does  not 
have  enough  time  in  their  busy  day  to 
design  their  own  exercises  and  assess- 
ment instruments.  Many  of  the  exercises 
included  in  this  kit  can  also  serve  as  a 
model  to  those  interested  in  designing 
their  own  instruments. 

The  kit  is  stored  in  an  attractive, 
green,  3-ring  binder  so  that  each  exercise 
can  easily  be  photocopied.  It  is  also 
divided  into  seven  units  with  several 
subsections: 

1.  Goals 
Reading 
Writing 
Math 

2.  Tutoring  or  working  in  groups 

Working  with  a tutor 

Working  in  a group 

3.  Skills 

Around  the  house 

Getting  around,  transportation 
Shopping 


Banking 

Looking  after  children 
Health 

Finding  a place  to  live 

Jobs 

4.  Learning  about  how  to  learn 

When,  where,  and  what  do  you  like  to  read 
and  write 
How  do  you  learn 
Thinking  about  what  you  read 

5.  Reading  and  writing  skills 
Starting  to  read  and  write 
How  are  you  doing  at  reading? 

How  are  you  doing  at  writing? 

Using  the  alphabet 

The  sounds  of  letters 
The  parts  of  words 
Punctuation 
Numbers 

6.  Making  changes 

Are  you  changing? 

Sharing 

Actions  for  change 

7.  Other  stuff 
Further  readings 

Thanks 

Programs  and  contacts 

Each  unit  offers  a good  transition  into 
the  next  unit,  although  I am  not  so  sure 
that  the  particular  order  is  the  one  I 
would  necessarily  follow  with  my 
learners.  But  most  teachers  would  not 
use  the  whole  kit  and  probably  just  pick 
and  choose  those  that  seem  to  fit  their 
learners'  basic  needs.  For  that  particular 
purpose,  I would  highly  recommend 
This  is  not  a test.  BT 
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TO  THE  COMMUNITY  LEARNING  CENTER: 

LOOK  AT  YOUR  STRENGTHS,  TOO! 

As  SABES'  Program  Development  Coordinator,  I was  thrilled  to  see  Mina  Reddy's 
article,  "Program  Evaluation  at  the  Community  Learning  Center,"  in  Volume  3 of 
Adventures  in  Assessment.  The  Community  Learning  Center's  experience  demonstrates 
much  of  the  flavor  of  SABES'  Integrated  Program  and  Staff  Development  Process.  The 
CLC  took  a systematic  approach  to  assessing  its  needs  and  strengths,  it  involved  staff 
and  students,  and  it  looked  beyond  the  traditional  time  frame  of  one  year. 

I would  like  to  encourage  the  Community  Learning  Center  to  focus  more  on 
strengths  just  as  it's  a good  place  to  begin  with  learners,  it's  a good  place  to  begin 
with  programs.  The  CLC  addressed  this  only  briefly,  "typing  uplhe  separate  action 
plans  into  one  document  and  prefacing  it  with  a statement  of  strengths  that  were 
identified  in  the  process.  Most  responses  to  both  evaluations  were  very  positive,  and 
this  was  not  acknowledged  anywhere  in  this  process,  which  focused  on  needs  and 
areas  for  development."  In  SABES'  process,  both  individual  strengths  and  program 
strengths  are  identified,  to  reflect  SABES'  philosophy  of  working  from  strengths,  and 
because  we  are  always  seeking  to  identify  and  make  available  to  everyone  the  exten- 
sive expertise  found  in  ABE  in  Massachusetts. 

More  than  30  programs  across  the  state  have  been  trained  in  the  Program  and  Staff 
Development  Process,  and  each  one  adapts  the  process  to  suit  its  philosophy  and 
needs.  The  Community  Learning  Center  provided  us  with  a good  example  of  how 
the  process  can  be  modified.  The  Integrated  Staff  and  Program  Development  Process 
Training  will  be  offered  in  each  region  again  soon;  programs  that  are  interested  in 
finding  out  more  about  the  training  process  should  contact  their  SABES  Regional 
Coordinators. 

Barbara  Garner 

Program  Development  Coordinator 
SABES 
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LETTER  FROM  URUGUAY 


Ms.  Loren  McGrail 
World  Education /SABES 
210  Lincoln  Street 
Boston,  MA  02111 

U.S.A.  December  9, 1 992 


Dear  Loren: 

You  don't  probably  remember  me,  except  for  the  fact  that  you  generously  gave  me 
an  issue  of  Adventures  in  Assessment  in  your  workshop  presentation  about  "Assess- 
ment in  the  Learner-directed  Adult  ESOL  class"  in  the  TESOL  Conference  in 
Vancouver  last  March. 

I remember  having  promised  you  some  sort  of  feedback  and  a copy  of  possible 
materials  that  could  be  generated  as  a result  of  reading  your  material. 

Congratulations  on  the  quality  and  nature  of  your  perspectives.  Your  publication 
has  given  us  a lot  of  impetus  to  go  on  exploring  the  assessment  issues.  We  have 
benefitted  from  materials  in  the  Appendix  as  well  and  several  workshops  have  been 
offered  having  as  backbones  your  checklist  and  questionnaires. 

I'm  sending  you  three  papers  that  were  prepared  by  teachers  in  the  workshops  I 
mentioned.  Feel  free  to  use  them  if  they  can  be  of  any  help  to  you. 

Sincerely  yours, 

Maria  Elena  Perera 

Pedagogical  Orientator 

Alianza  Cultural  Uruguay  - Estados  Unidos  de  America 


[Editor's  note:  The  following  assessment  tools  focus  on  listening  skills.  We  encourage  readers 
to  contribute  other  ESL  or  adult  learning  materials  on  this  infrequent  but  rich  topic.] 
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LISTENING  PROGRESS 

Vocabulary 

Is  the  vocabulary  new? 

Does  it  hinder  your  understanding  of  the  whole  passage? 

Compensation 

Is  it  difficult  for  you  to  guess  the  meaning  of  some  of  the 
new  words? 

Short  Term  Memory 

How  many  times  do  you  feel  you  need  to  listen  to  a new 
passage/ conversation? 

Grammar/Value  of  Utterances 

After  listening  to  the  passage,  can  you  briefly  paraphrase 
it?  Or  while  you  are  listening  to  it,  can  you  take  some 
notes? 

Speed 

Did  the  speakers  go  too  fast  for  you  to  keep  up  with  their 
pace? 

Did  the  speed  really  interfere  with  your  understanding  of 
the  whole? 

External  References 

Were  the  speakers  just  informing,  or  were  they  also  giving 
opinions  (implicitly)? 

NOTE:  These  questions  could  be  answered  checking  a scale  (1  to  S)  or  checking  * Not  at  alt*  " Partially / or  “Almost  completely :m 

1.  Could  you  follow  the  conversation  in  a successful  way? 

2.  Were  you  able  to  understand  most  of  the  words? 

3.  How  did  you  do  it? 

a.  guessing  the  meaning  of  unknown  words? 

b.  guessing  the  meaning  of  words  you  didn’t  hear? 

c.  predicting  according  to  the  interpretation  of  the 
picture? 

d.  other? 

4.  How  was  the  speed  of  the  conversation? 

Slow 

Medium 

Medium-Fast 

Fast 


Not  at  allt  I got  it 1 
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so  LETTERS 


1 got  it! 

Not  at  alii 

5 

4 

3 

2 

1 

1.  Prediction  helped  me. 

2.  1 understood: 

a.  the  general  meaning 

b.  the  details 

c.  the  purpose. 

3.  Because  of  the  speed  1 could  understand. 

4.  1 could  use  the  context  to  guess  at  unfamiliar  or  unheard 
words. 
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ON  PORTFOLIO  ASSESSMENT 

Thanks  for  your  work  on  Adventures  in  Assessment.  I have  been  working  on  a 
project  for  the  U.S.  Department  of  Education,  Division  of  Adult  and  Vocational  Educa- 
tion, on  portfolio  assessment  for  the  past  few  months.  In  the  course  of  this  project.  I've 
talked  with  people  working  in  volunteer,  adult  basic  education,  state  and  federally- 
funded,  workplace  literacy,  English  as  a Second  Language,  job  training,  grassroots 
community-based,  public  school  and  community  college,  and  family  literacy  pro- 
grams. I spoke  with  teachers,  volunteers,  staff  development  specialists,  program 
administrators,  and  state  directors.  Everyone  is  excited  about  the  potential  of  alterna- 
tive and  portfolio  assessment,  but  few  feel  that  they  know  how  to  approach  it. 

I talked  with  some  people  who  began  to  implement  portfolio  assessment  as  isolated 
individuals  because  they  had  read  about  it  and  they  were  intrigued,  or  because  they 
used  it  in  their  public  school  teaching  with  children.  In  most  cases,  however,  I found 
that  people  experienced  with  portfolio  assessment  exist  in  local  groups,  and  were 
introduced  to  portfolio  assessment  through  some  kind  of  organized  staff  development 
activitv. 

Many  of  those  I spoke  with  who  are  engaging  in  portfolio  assessment  know  of  other 
practitioners  in  their  local  area  who  are  also  using  portfolios.  However,  they  often 
were  surprised  by  my  telephone  call  and  thirsty  for  news  of  how  portfolio  assessment 
was  being  implemented  in  other  parts  of  the  country.  They  wanted  things  to  read, 
people  to  talk  to,  and  other  models  to  look  at.  Many  mentioned  Adventures  in  Assess- 
ment as  one  of  a very  few  things  available  that  addresses  alternative  assessment  in 
adult  literacy  education  specifically.  The  literature  in  portfolio  assessment  for  those 
who  teach  children  is  often  quite  applicable  to  adult  literacy  practitioners.  However, 
it  still  requires  some  translation  process,  and  it  does  not  help  adult  literacy  educators 
develop  a sense  of  belonging  to  a larger  group  that  is  struggling  with  similar  issues. 

It  is  clear  that  we  need  to  develop  a number  of  mechanisms  to  help  practitioners 
who  are  implementing  portfolio  and  other  forms  of  alternative  assessment  in  adult 
literacy  education  to  continue  learning  and  to  share  their  experiences  and  resources. 
We  need  materials  that  address  alternative  assessment  broadly,  placing  portfolio 
assessment  and  other  approaches  within  a larger  theoretical  framework.  We  need 
how-to's  as  well  as  conceptual  explorations.  We  need  many  more  publications  that 
examine  and  share  practitioners'  experiences.  We  also  need  mechanisms  such  as 
computer  bulletin  boards  that  encourage  active  sharing. 

Leadership  development  in  relation  to  alternative  assessment  has  to  be  a priority; 
the  influence  of  those  who  have  been  providing  professional  development  services  in 
this  area  can  be  clearly  seen. 

All  of  us  have  to  advocate  for  the  incorporation  of  portfolio  and  other  forms  of 
alternative  assessment  into  the  indicators  of  program  quality  currently  being  devel- 
oped in  each  state.  Model  Indicators  of  Program  Quality  for  Adult  Education  Programs, 
released  by  the  U.S.  Department  of  Education,  Office  of  Adult  and  Vocational  Educa- 
tion (1992),  identifies  portfolio  assessment  as  a sample  measure  for  showing 
educational  gains  (Indicator  1).  However,  this  guide  illustrates  that  portfolio  assess- 
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merit  can  make  an  important  contribution  to  a number  of  other  indicators  as  well.  For 
example,  portfolio  assessment  has  the  potential  to  enable  practitioners'  development 
of  a much  more  concrete  and  comprehensive  knowledge  of  students'  goals,  interests, 
and  approaches  to  learning.  This  can  assist  in  program  planning  (Indicator  3),  curricu- 
lum and  instruction  (Indicator  4),  staff  development  (Indicator  5),  and  retention 
(Indicator  8).  An  article  on  influencing  state  and  national  policy  would  be  very  useful. 

And,  finally  we  must  continue  working  to  improve  professional  support  and  work- 
ing conditions  in  adult  literacy  education  as  a way  of  supporting  innovation. 
Implementing  portfolio  and  other  forms  of  alternative  assessment  depends  on  using 
whole  language,  participatory  approaches  to  instruction;  instructors  must  feel  com- 
fortable with  the  teaching  and  learning  process  in  order  to  integrate  assessment  with 
instruction.  Teachers  also  must  have  paid  preparation  time,  space  in  which  to  store 
folders  and  materials,  and  some  job  security  so  that  the  process  of  implementing 
portfolio  assessment  can  be  honored. 

I know  that  these  ideas  are  not  new;  however,  I think  we  need  to  revisit  them 
continually  as  we  try  to  move  forward.  I appreciate  your  work  on  Adventures  in 
Assessment,  and  I look  forward  to  a larger  continuing  conversation. 

Hanna  Fingeret 

Executive  Director,  Literacy  South 
Durham,  North  Carolina 


AN  INVITATION  TO  WRITE 
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Adventures  in  Assessment  accepts  articles  regarding  any  of  the  three  compo- 
nents of  assessment:  Getting  Started,  Ongoing,  and  Looking  Back,  or  our  other 
departments:  What  Counts?,  Voices  from  the  Field,  and  Publication  Reviews. 

If  you  would  like  to  submit  an  article  to  Adventures  in  Assessment, 
please  contact  us  at: 

Adventures  in  Assessment 
Loren  McGrail,  editor 
SABES 

210  Lincoln  Street 
Boston,  MA  02111 
617-482-9485 

The  next  issue  of  Adventures  in  Assessment  will  also  feature  readers' 
responses  to  the  articles  in  this  and  earlier  volumes. 

Perhaps  one  of  the  articles  reminded  you  of  a classroom  assessment  problem 
you  faced  and/or  solved.  Maybe  you  have  questions  or  additional  information 
about  the  tools  and  procedures  you  read  about  Or  perhaps  you  just  feel  like 
conversing  with  other  practitioners  who  are  looking  at  issues  of  alternative 


We  welcome  your  input  and  feedback.  To  be  included  in  the  Fall  Adventures 
in  Assessment,  please  send  your  letters  no  later  than  July  1, 1993  to  Loren 
McGrail  at  the  above  address. 
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System  tor  Adu*  Basic  Education  SupDcrt 


SABES  is  the  System  for  Adult  Basic  Education  Support,  a comprehensive  training  and  technical 
assistance  initiative  for  adult  literacy  educators  and  programs . Its  goal  is  to  strengthen  and  enhance 
literacy  services  and  thus  to  enable  adult  learners  to  attain  literacy  skills.  SABES  accomplishes  this 
goal  through  staff  and  program  development  workshops , consultation,  mini-courses,  mentoring  and 
peer  coaching,  and  other  training  activities  provided  by  five  Regional  Support  Centers  located  at 
community  colleges  throughout  Massachusetts . 

SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect , evaluate , and  disseminate  ABE 
materials,  curricula,  methodologies,  and  program  models . Each  of  the  five  SABES  Regional  Support 
Centers  similarly  offers  program  support  and  a lending  library.  SABES  maintains  an  Adult  Literacy 
Hotline , a statewide  referral  service  which  responds  to  calls  from  new  learners  and  volunteers.  The 
Hotline  number  is  1-800-447-8844. 

The  SABES  Central  Resource  Center,  located  at  World  Education,  publishes  a statewide  quarterly 
newsletter,  “ Bright  Ideas,  " and  journals  on  topics  of  inieresi  10  aauh  'literacy  professionals,  such  as 
this  volume  of  “Adventures  in  Assessment.  ” 

The  first  three  volumes  of  “Adventures  in  Assessment"  present  a comprehensive  view  of  the  state  of 
practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners.  Volume  1,  Getting 
Started,  included  start-up  and  intake  activities.  Volume  2,  Ongoing,  shares  tools  for  ongoing 
assessment  as  part  of  the  learning  process;  Volume  3,  Looking  Back,  Starting  Again,  focused  on  tools 
and  procedures  used  at  the  end  of  a cycle  or  term,  including  self  class . and  group  evaluation  by  both 
teachers  and  learners.  Volume  4 covered  a range  of  interests.  Volume  5,  Tale  of  the  Tools  was 
dedicated  to  reflecting  on  component  3 tools  of  alternative  assessment.  Volume  6,  Responding  to  the 
Dream  Conference,  was  dedicated  to  responses  to  Volumes  1-5.  Volume  7,  The  Partnership  Project, 
highlighted  writings  from  a mentoring  project  for  practitioners  interested  in  learning  about 
participatory  assessment.  Volume  8 covered  a range  of  topics  from  education  reform  to  learner 
involvement  in  assessment.  Volume  9 looked  at  assessment  in  a volunteer  program,  native  language 
literacy,  ESOL  programs,  and  evaluations  in  workplace  education  programs. 

We  would  like  to  see  your  contribution,  especially  if  you  are  doing  native  language  literacy  or  low- 
level  ESOL  literacy.  Contact  editor,  Alison  Simmons,  to  discuss  your  submission . 

Permission  is  granted  to  reproduce  portions  of  this  journal.  We  request  appropriate  credit  be  given 
to  “Adventures  in  Assessment"  and  to  the  author. 

“Adventures  in  Assessment"  is  free  to  DOE-fimded  Massachusetts  programs;  out-of-state  requests 
will  be  charged  a nominal  fee,  at  cost.  Please  write  to,  or  call: 

Alison  Simmons 
SABES  Central  Resource  Center 
World  Education 
44  Farnsworth  Street 
Boston,  MA  02210-1211 
617-482-9485 
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The  Tale  of  the  Tools 

A Matter  of  Stance  and 

Dance 

Learners  Logs  will  never  be  a finished 
product. 

Gear 

Examining  inconsistent  results,  my  col- 
leagues and  I reconsidered  the  tools  . 

Mueller 

I am  still  uncertain  about  the  best  ways  to 
document  the  development  of  self-esteem  and 
community,  but  I do  feel  that  my  teacher  log 
will  play  an  important  part. 

Barn/ 

Obviously  nothing  that  is  based  on  human 
beings'  changing  needs  and  goals  is  ever 
finished  or  perfect. 

Kelly 

I have  a lot  more  questions  than  answers.  In 
fact  I am  going  to  be  revealing  a lot  of  (gulp) 
mistakes , which  Lord  knows  teachers  do  not 
really  make. 

Trunnell 


by 

Loren  McGrail 


Editor 

SABES  Central 
Resource  Center/ 
World  Education 


Never  finished  or  perfect,  uncer 
tainty,  inconsistent  results,  more 
questions  than  answers.  These  are  just  a 
few  of  the  voices  of  reflection  from  the 
authors  who  participated  in  Component 
#3  of  the  Greater  Opportunities  in  Adult 
Learners  Success  (G.O.A.L.S.)  Project 
developed  by  Sandy  Brawders  for  the 
Bureau  of  Adult  Education  and  coordi- 
nated first  by  Lindy  Whiton,  then 
Charlotte  Baer  and  Caroline  Gear.  Each 


of  the  programs  involved  in  this  project 
developed  assessment  and  evaluation 
tools  that  were  based  on  educational 
theories  or  approaches,  on  who  their 
learners  and  teachers  were  and,  finally, 
on  what  the  goals  and  interests  of  the 
learners  were  and  how  they  used  literacy 
in  their  daily  lives. 

These  reflective  writings  which  are 
the  focus  of  this  volume  of  Adventures  in 
Assessment  are  documentations  of 
inquiry,  teachers  researching  their 
practice.  As  Whiton  says  about  the 
writings  in  Adventures  in  Assessment  in 
general,  "This  is  the  place  where  other 
teachers  can  listen  not  only  to  the  ‘"tale  of 
the  Tools"  but  to  the  narrative,  the  story  of 
who,  why,  and  how  come"  (Whiton,  Docu- 
menting the  Voices,  paper  delivered  at 
AAACE,  1993). 

Just  as  authentic  assessment  asks 
learners  to  develop  the  habit  of  pausing 
to  reflect  before  moving  on,  so  too  must 
teachers  adopt  the  practice  of  "taking 
time  to  stop  and  think,  to  observe  and  make 
sense  of  the  activities  and  progress  of  their 
students"  (Lessoules  and  Gardner,  1992). 
This  sentiment  is  echoed  by  Stephen 
Brookfield  when  he  says,  "Becoming 
skillfull  teachers  is  a matter  of  stance  and 
dance.  It  is  a process  that  involves  us  taking 
a critically  reflective  stance  on  practice, 
studying  our  actions  and  reasoning  about 
teaching  as  open-ended,  problematic  phe- 
nomena. It  also  engages  us  in  a dance  of 
experimentation  and  risk-taking  in  which  we 
explore  how  learners  experience  learning 
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while  living  out  our  pedagogical  convictions 
and  trying  to  realize  democratic  values  in 
our  practice  (Brookfield,  1993). 


I want  to  understand  the  students  progress. 
1 want  to  see  and  1 want  the  students  to  see 
the  complicated  reality  of  learning. 

T runnel 


DOCUMENTING  LEARNING: 

WHAT  THE  LEARNERS  GET 
How  learners  experience  learning  is  a 
theme  that  reoccurs  throughout  all  these 

. i / -l • r v / n — A 

leacners  aiuiies*.  rui  iviucxici  hi 

ing  Self  Evaluation  Checklists:  A Starting 
Point  for  Dialogue,  the  concern  is  how  can 
students  take  responsibility  for  their 
own  learning?  Her  careful  analysis  of  the 
data  her  checklists  revealed  about  Maria 
and  Carlotta  lead  her  to  ask  "Is  it  impor- 
tant that  students  move  in  a linear 
progression  in  their  language  acquisition  or 
English  usage?”  This  question  is  of 
particular  importance  to  teachers  work- 
ing with  ESOL  students  who  are 
interested  in  documenting  acquisition, 
not  just  language  learning.  This  question 
also  reflects  Mueller's  understanding 
that  a lack  in  linear  progress  may  also 
indicate  a significant  difference  in  how 
learners  perceive  their  own  progress.  "If 
big  gaps  exist  between  their  perceptions  of 
their  abilities  and  ours,  then  it  is  important 
to  dig  deeper  into  the  'why'."  The  self- 
evaluation  checklist  then  becomes  the 
starting  point  for  dialogue,  not  the  end 
point 

For  Bany,  writing  about  and  with  her 
student  Pat  in  Reflections  on  On-going 
Assessment:  Documenting  Self-Esteem  and 
More,  the  issue  of  how  learning  is  experi- 
enced was  focused  on  how  Pat  felt  she 


was  making  progress  and  how  she  knew 
when  she  was.  Barry's  main  concern  was 
that  while  Pat's  development  of  self- 
esteem and  sense  of  community  were 
apparent  from  her  accomplishments  and 
conversations  with  Barry,  the  tools  and 
procedures  used  in  the  program  did  not 
reflect  this  back  to  the  learner.  Barry' s 
reflection  is  a critical  looking  back  at 
what  her  tools  both  documented  and 
revealed  to  her  as  a teacher  and  what 
they  revealed  or  did  not  reveal  to  her 
student  Pat. 

Barry's  comment  "We  need  to  have  more 

/*/w <t i />w /rf ■» /-» v»  /llintf#  lammivift  tf fA— 
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ments  and  more  time  together  to  analyze  the 
data  gained  from  these  checklists"  is  echoed 
by  many  authors  but  probably  most 
strongly  by  veteran  tool  maker  and 
mentor,  Janet  Kelly.  In  A Reflection  on  the 
Ideal  vs  the  Real  Kelly  admits,  "I  think  we 
have  relied  more  on  what  we  as  teachers 
thought  was  important  to  learners,  what  we 
were  interested  in  knowing  about  their 
progress,  what  we  thought  we  could  docu- 
ment, and  what  we  thought  funders  would 
want  to  know." 

She  goes  on  to  say  that  these  beliefs 
still  form  the  basis  of  choices  in  assess- 
ment practices  but  that  she  hopes  the 
program  will  find  ways  "to  invite  learners 
to  share  more  effectively  in  decisions  not 
only  about  program  and  curriculum  evalua- 
tion but  also  in  the  selection  of  and  critique 
of  materials  for  their  own  assessment  portfo- 
lios." 

Like  Barry,  Kelly's  reflection  piece 
also  examines  and  critiques  previously 
published  tools  at  the  Read/Write/Now 
Program  and  discusses  the  development 
of  new  tools.  Of  particular  note  is  the 
discarding  of  learning  logs  (a  tool  much 
favored  in  Barry's  program)  in  favor  of  a 
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His  use  of  the 
teacher  log  dem- 
onstrates one  of 
the  key  principles 
of  learner-centered 
assessment  — 
assessment  must 
feed  back  to 
instruction. 


shared  teacher's  log  where  " Significant 
anecdotes  and  observations  could  be  recorded 
on  a joint  teacher  [learner  from  each  learning 
contract  period  or  month."  Kelly's  experi- 
ence, like  Barry's,  reminds  us  that  what 
seems  to  work  best  is  time  and  trust. 

"The  longer  learners  are  part  of  the  learning 
center  community,  the  better  they  get  at 
telling  us  what  works  for  them,  what  does 
not,  what  they  like  and  do  not  like  and  why” 
Trunnel,  in  Danger:  Road  Construction 
Ahead,  like  Barry  and  Kelly,  is  also 
concerned  with  how  the  learners  experi- 
ence the  tools  he  has  created.  "If  you 
asked  them  about  these  tools,  they  would 
likely  have  much  different  responses  and 
potentially  much  different  assumptions 
about  what  should  be  assessed  and  what 
indicators  they  need."  Similarly  he  finds 
the  teacher  log  to  be  an  invaluable  tool 
that  records  more  completely  the  emo- 
tional and  highly  complicated  task  that 
learning  is  beyond  skill  acquisition.  His 
use  of  the  teacher  log  demonstrates  one 
of  the  key  principles  of  learner-centered 
assessment  — assessment  must  feed 
back  to  instruction.  Trunnel  suggests 
that  the  notes  he  keeps  are  also  for  his 
own  improvement,  what  strategies  he 
has  used,  what  has  worked,  what  has 
not.  He  even  suggests,  "Perhaps  there  is 
a way  it  can  be  used  too  in  teacher- 
sharing at  my  center  or  in  a teacher 
self-evaluation."  Critical  self  reflection  is 
a hallmark  of  all  these  reflective  writ- 
ings, however  Trunnel  stands  alone  in 
his  ability  to  see  and  then  identify  the 
different  kinds  of  checklists,  "The  check- 
lists that  I use  are  skilb  based,  not  strategy 
based,  and  thus  do  not  record  all  the  infor- 
mation I want." 


Like  Trunnel,  Gear  in  writing  about 
The  Learning  Log:  Evolution  of  an  Assess- 
ment Tool  was  interested  in  how  to  create 
a tool  that  recorded  information  that  was 
useful  to  the  teacher  and  the  student 
alike.  Her  needs  and  questions  are 
different  but  just  as  important:  "How  can 
we  get  students  to  look  at  their  own  learn- 
ing? How  can  we  see  their  learning  as  a 
process  and  evaluation  of  their  progress  as 
part  of  the  process  ? How  can  we  get  the 
students  to  measure  their  own  progress 
rather  than  rely  solely  on  the  instructor? 
How  can  we  get  our  students  to  answer  the 
weekly  evaluation  questions  with  more  than 
one  sentence?  How  can  we  get  our  students 
to  critique  the  classes  and  know  it  isn't  going 
to  be  taken  as  an  insult  to  the  instructor? 

Revising  tools  was  a similar  concern 
for  Oesch.  In  Adapting  Tools  To  A New 
Program,  Oesch  describes  how  tools 
designed  for  one  program  were  revised 
or  adapted  to  another  at  the  same 
agency.  Some  of  you  familiar  with 
Martha  Germanoski's  article,  in  the  first 
issue  of  Adventures  in  Assessment,  will 
recall  her  goals  checklist  and  the  daily 
log.  These  tools  have  been  adapted  and 
revised  to  assist  learners  in  taking 
responsibility  for  their  learning,  not  fix 
the  learner  but  help  the  learner  build  on 
his/her  strengths.  Oesch  also  states  that 
the  revision  in  the  EGAP  (Educational 
Goals  Assessment  Package)  led  to 
changes  in  the  curriculum.  This  is  a good 
example  of  how  a change  in  the  assess- 
ment "tail"  can  wag  the  curriculum 
"dog." 
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DOCUMENTING  COMMUNITY 
DEVELOPMENT  INSIDE  AND 
OUTSIDE  THE  CLASSROOM 
For  Judy  Hofer  and  Pat  Larson  in 
Reflecting  on  the  Links  Between  Literacy 
Practices  and  Community  Development,  the 
issue  of  building  on  learner  strengths  is 
a collective  endeavor.  "We  want  to  set 
forth  right  from  the  start  that  people  coming 
to  an  adult  education  program  must  take  an 
active  role  in  their  own  learning,  and  that 
working  within  a group  will  enhance  that 
process...  Getting  a GED  and  improving 
reading  and  writing  skills  barely  scratch  the 
surface  ofwhai  is  needed  for  people  to  get  out 
of  poverty  and  exercise  more  control  over 
their  lives."  Hofer  and  Larson  are  con- 
cerned with  bringing  the  larger 
community  into  the  classroom  and  the 
classroom  to  the  larger  community  and 
how  to  create  tools  that  measure  not 
only  individual  progress  but  how  to 
assess  what  people  do  as  a group/ 
community.  They  have  benefited  and 
borrowed  from  Sondra  Stein's  Framework 
for  Assessing  Program  Quality  designed 
for  community-based  programs  for  the 
Association  for  Community-based 
Education.  This  framework  and  a view 
of  literacy  as  practice  helped  them  to 
generate  a list  of  guiding  questions  that 
guided  their  inquiry.  Some  of  the  ques- 
tions are  as  follows:  How  do  we  assess 
improvement  and  change  as  a continuous 
process  not  only  for  the  individual  but  for 
the  community  of  individuals?  How  do  we 
assess  if  there  is  movement  or  change  on  a 
continuum  in  terms  of  community  develop- 
ment by  people  participating  in  programs 


and  by  the  program  ? How  do  we  develop  a 
language  of  assessment  which  takes  into 
account  the  development  of  community  and 
collaborative  efforts  at  adult  education 
centers  which  also  speaks  to  the  needs  of 
funders? 

THE  STANCE  AND  THE  DANCE 

Becoming  skillful  teachers  in  learner- 
centered  approaches  to  assessment  is  a 
matter  of  stance  and  dance.  It  is  a pro- 
cess that  requires  us  to  take  a reflective 
stance  on  our  practice  (both  past  and 
present),  to  be  clear  about  what  we  stand 
for  or  arc  trying  tc  achieve,  yet  tc  pose 
questions  rather  than  seek  simple  solu- 
tions, to  be  open  to  engage  in  a dance  of 
experimentation  and  revision,  and 
finally  to  risk  finding  out  not  just  how 
our  learners  experience  learning  but  how 
we  help  or  fail  them  in  that  process.  The 
G.O.A.L.S.  Project  allowed  a handful  of 
practitioners  in  the  state  of  Massachu- 
setts the  resources,  time,  and  money  to 
carefully  map  their  route  and  compare 
their  travels.  The  toolkit,  according  to 
Trunnel,  is  the  "Atlas,  that  we  hope  will 
help  others  explore  other  roads  and 
paths."  To  make  the  trip  a little 
smoother,  we  have  included  these 
reflective  writings  as  a kind  of  legend  to 
understanding  why  some  roads  are 
paved  while  others  are  still  dirt  roads. 

As  editor  of  Adventures  in  Assess- 
ment, I invite  you  all  to  contribute  your 
reflections  or  reactions  so  that  we  may 
learn  from  each  other,  discover  new 
roads  together. 


Becoming  skillful 
teachers  in 
learner-centered 
approaches  to 
assessment  is  a 
matter  of  stance 
and  dance.  It  is 
a process  that 
requires  us  to  take 
a reflective  stance 
on  our  practice 
(both  past  and 
present),  to  be 
clear  about  what 
we  stand  for  or  are 
trying  to  achieve, 
yet  to  pose 
questions  rather 
than  seek  simple 
solutions. 


Brookfield,  Stephen.  “Recognizing  Skillful  Teaching,”  unpublished  manuscript,  1993. 

Zessoules,  R.  and  Gardner,  H.  “Authentic  Assessment  Beyond  the  Buzzword  and  Into  the  Classroom,”  in  Perron  e, 
V.  (ed),  Expanding  Student  Assessment,  1 991 . Alexandria,  VA:  Association  for  Supervision  and  Curriculum  Develop- 
ment, p.  47-71. 
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The  Alternative  Assessment  Aspect  of  the  G.O.A.L.S.  Project 

The  Process  of  Component  #3 


by 

Lindy  Whiton 


Western  Mass.  SABES; 
former  Coordinator, 
Component  #3 


This  issue  of  Adventures  in  Assess- 
ment is  dedicated  to  discussing  the 
process  a group  of  teachers  involved  in 
alternative  assessment  went  through 
over  the  past  year.  The  group  that  put 
together  the  articles  contained  here  were 
originally  drawn  together  to  complete 
part  of  the  Greater  Opportunities  in 
Adult  Learners  Success  (G.O.A.LS) 
Project. 

The  original  intention  of  the 
G.O.A.LS.  Project,  which  was  developed 
by  Sandy  Brawders  for  the  Bureau  of 
Adult  Education,  was  to  investigate  and 
develop  an  accountability  system  which 
would  be  a true  reflection  of  the  field, 
one  that  intrinsically  asks  teachers, 
directors  and  learners  their  opinion  on 
"What  are  ya  doin'?  What  would  be 
helpful?  When  are  you  having  prob- 
lems? How  do  you  make  changes?  Did 
you  discover  anything  new  this  year  that 
you  would  like  to  share?" 

The  G.O.A.LS.  Team  interviewed  the 
staff  of  more  than  70  programs.  We 
talked  to  both  teachers  and  directors.  We 
interviewed  250  learners,  both  ABE  and 
ESL  (see  Adventures  in  Assessment,  VoL 
1).  We  asked  the  field  to  be  a part  of  a 
discovery  process.  Although  this  project 
has  come  from  the  Bureau  of  Adult 
Education  and  not  the  field,  it  has 
adopted  several  different  techniques  to 
get  at  the  language  the  field  uses,  to  get 
at  the  needs  of  the  field,  to  find  a true 
reflection  and  voice  of  the  field. 

Although  the  original  intention  of 


gathering  this  group  together  was  not 
for  staff  development  nor  as  a teacher 
research  project,  it  has  been  both.  What 
resulted  was  both  good  staff  develop- 
ment and  a form  of  teacher  research. 

THE  ALTERNATIVE  ASSESSMENT 
ASPECT  OF  G.O.A.L.S. 

The  objectives  of  the  G.O.A.L.S. 
project  were  as  follows: 

• Have  an  accountability  system  that 
reflects  the  entire  field; 

• Provide  an  opportunity  for  all  voices 
to  be  heard; 

• Give  individuals  an  opportunity  to 
impact  on  state  policy  making; 

• Develop  a leamer-centered  account- 
ability system1; 

• Look  for  an  organic  language  of 
evaluation  that  is  accessible  to  the 
learner,  not  ‘secret  knowledge’  only 
decipherable  to  bureaucrats; 

• Be  able  to  pay  for  what  it  asks; 

• Create  an  appropriate  framework, 
rather  than  rely  on  already-existing 
frameworks  that  may  be  inadequate; 
This  project  assumes  a staff  develop- 
ment network  and  system  in  place 
where  there  is  support  for  change 
(training)  in  the  field  (SABES). 

Component  #3  is  the  section  of  the 
G.O.A.L-S.  project  that  looks  at  alterna- 
tive means  of  assessment.  The  Depart- 
ment of  Education  chose  to  run  this 
component  as  a teacher-based  research 
project  to  develop  a state  run  account- 
ability system,  one  that  would  foster  a 
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dialogue  with  adult  learners,  teachers, 
and  other  staff,  legislature,  the  federal 
government  and  the  public  at  large. 

The  questions  that  were  important  to 
us  throughout  the  entire  project  were: 

• What  must  the  Bureau  know  in  order 
to  fund  successful  programs? 

• How  does  the  Bureau  open  lines 
between  programs  and  the  Bureau  so 
that  learners  benefit  from  everyone's 
expertise? 

• What  do  the  programs  need  from  the 
Bureau  to  best  do  their  jobs?  • 

• Can  we  develop  a system  of  account- 
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and  richness  of  programs,  thus  reflect- 
ing the  variety  and  richness  of 
learners? 

• In  Component  #3,  can  we  develop  a 
document  that  similarly  represents 
our  variations  and  that  allows  for  one 
piece  of  the  puzzle  to  be  placed,  so 
that  when  we  step  back  and  look  at  till 
the  components  as  a whole,  alterna- 
tive assessment  is  clear  and  helpful? 

The  majority  of  the  participating 
programs  already  had  in  place  curricula 
that  reflected  the  principles  of  alterna- 
tive assessment,  and  were  attempting  to 
develop  tools  to  inform  teachers  and 
learners  as  to  the  effect  of  these  cur- 
ricula. (See  Adventures  in  Assessment, 
Volume  #4  - Whiton  and  McGraiL)  This 
group  believed  learners'  goals  were  not 
only  important  but  a driving  force  in 
curriculum,  thus,  these  assessment 
practice  are  based  on  the  individual 
goals  of  the  learner  as  well  as  the  goals 
of  the  individual  programs.  Whether  we 
are  teaching  ABE,  GED,  or  ESL  literacy 
practices  have  to  directly  relate  not  only 
to  the  goals  of  learners,  but  to  their 
immediate  lives  as  well.  Our  beliefs  are 


backed  by  the  works  of  Lytle  (1989, 

1990),  Auerbach  (1990),  Spruck  Wrigley 
(1992)  and  Fingeret  (1989).  To  discuss 
learners'  growth  by  talking  about  stan- 
dardized increases  in  reading  levels 
"ignores  other  legitimate  criteria  for 
evaluating  a literacy  program,  like  the 
quality  of  the  curriculum,  teaching,  or  its 
connection  to  significant  social  issues 
relevant  to  students'  lives  and  interests; 
and,  it  fails  to  recognize  that  increases  in 
reading  scores  have  little  to  do  with  the 
way  adults  live  and  use  literacy  in  the 
real  world"  (McGrail  1991).  And  with 
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assessments  "fail  to  distinguish  between 
language,  literacy,  and  culture;  that  is, 
they  don't  tell  us  whether  the  learner  has 
trouble  with  an  item  because  he  or  she 
(a)  is  unfamiliar  with  the  cultural  notion 
underlying  the  task  (b)  lacks  the  requi- 
site knowledge  of  English  vocabulary  or 
sentence  structure  or  (c)  does  not  have 
enough  experience  with  reading  and 
writing  to  complete  the  task"  (Spruck 
Wrigley,  1992). 
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organic  language 
of  evaluation  that 
is  accessible  to  the 
learner,  not  'secret 
knowledge ' only 
decipherable  to 
bureaucrats 


Reading  assessment  must  reflect  recent 
advances  in  the  understanding  of 
reading  process.  The  International 
Reading  Association  is  concerned 
that  instructional  decisions  are  too 
often  made  from  assessments  which 
define  reading  as  a sequence  of  discrete 
skilb  that  students  must  master  to 
become  readers.  Such  assessments  foster 
inappropriate  instruction .2 


Thus,  based  on  the  educational 
theories  that  drive  the  program's  cur- 
riculum, each  of  the  programs  involved 
in  this  project  developed  assessment/ 
evaluation  tools  that  consider  who  their 
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It  was  fascinating 
to  see  the  variety 
of  tools.  Even 
more  impressive 
were  the 
variations  on 
the  same  tools. 
Teachers  had 
adapted  models  to 
make  tools  that 
were  appropriate 
for  their  sites  and 
needs. 


learners  and  teachers  are,  the  goals  and 
interests  of  their  learners,  and  how  they 
used  literacy  in  their  daily  lives. 

The  development  of  a community  of 
learners  is  an  extremely  important  factor 
in  the  success  of  learners'  reaching  their 
goals  and  our  curricula  and  our  assess- 
ments/evaluation tools  need  to  reflect 
this,  too.  These  practices  inspire  and 
cultivate  community  involvement  and 
ownership  of  educational  decisions  in 
the  classroom. 

The  articles  that  follow  reflect  the 
hard  work  of  these  programs.  They 
combined  what  they  knew  about  alterna- 
tive assessment  with  the  needs  of  the 
Bureau,  and  the  needs  of  other  pro- 
grams. These  articles,  or  notebooks,  are 
the  product  of  the  process  that  Compo- 
nent #3  went  through  while  wrestling 
with  the  issues  of  accountability,  assess- 
ment, and  evaluation.  They  don't  say, 
"Here,  just  duplicate  and  standardize 
these  tools."  Rather,  they  relate  the 
process  they  went  through  in  developing 
the  tools  to  help  other  practitioners 
develop  their  own  tools  based  on  their 
own  curricula,  learners,  and  classrooms. 

THE  PROCESS  OF 
ASSESSING  ASSESSMENT 

The  first  meeting  was  in  April,  1992. 
We  introduced  ourselves,  then  immedi- 
ately sat  down  to  the  task  of  defining 
assessment  and  evaluation.  First,  we 
attempted  to  get  the  entire  group  to 
settle  on  one  definition  of  assessment 
and  one  of  evaluation.  We  could  not.  We 
agreed  to  disagree.  (Since  then,  we  have 
reached  agreement.) 

At  the  end  of  the  first  meeting,  we 
decided  the  goal  of  the  next  meeting 
would  be  to  share  the  tools  the  group's 


members  had  already  developed  and  to 
describe  our  programs  to  each  other. 
This  was  done  in  early  June,  1992.  It  was 
fascinating  to  see  the  variety  of  tools. 
Even  more  impressive  were  the  varia- 
tions on  the  same  tools.4  Teachers  had 
adapted  models  to  make  tools  that  were 
appropriate  for  their  sites  and  needs.  By 
the  end  of  our  sharing,  we  knew  we 
needed  some  format  to  bring  us  all 
together.  I was  then  asked  to  write  a 
philosophical  statement  and  some  ideas 
for  direction,  distribute  this  paper  to 
everyone,  and  then  call  us  back  together. 
At  this  point  I developed  the  chart 
(Figure  1)  adapted  from  Rena  Sofer's 
work  (Sofer,  Rena,  Complete  Theory-to- 
Practice  Handbook ).  It  lists  all  the  different 
types  of  tools  that  had  been  adopted  by 
the  programs  and  fits  them  into  a frame- 
work that  addresses  accountability 
issues.  The  paper  also  divides  tools  into 
the  categories  "Starting  Up,"  "On- 
Going,"  and  "Looking  Back"  (Auerbach, 
McGrail,  Adventures  in  Assessment, 
Volume  1). 

At  our  next  meeting,  we  decided  to 
split  up  into  groups  that  represented 
these  three  categories.  These  groups 
would  develop  a tool  kit  including 
examples  of  tools  in  each  category  and 
describe  when,  why,  how,  and  who 
might  use  these  tools.  This  discussion 
raised  the  question  of  what  the  tool  kit 
had  to  do  with  accountability.  Could  the 
tool  kit  gather  the  type  of  information 
that  DOE  was  requesting?  It  was  felt  that 
for  the  majority  of  these  programs, 
reading  levels,  acquiring  a job,  or  enter- 
ing college  were  not  the  only  outcomes 
nor  were  they  the  only  measures  of 
success.  Actually,  in  terms  of  reading 
levels,  we  thought  just  the  opposite,  that 
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to  talk  about  reading  levels  was  irrel- 
evant to  literacy  programs.  The  types  of 
measures  we  felt  were  important  were 
not  addressed  by  either  state  or  federal 
reports. 

The  teachers  wanted  to  talk  about 
changes  in  attitudes  or  smiles,  involve- 
ment with  school  systems,  or  finding  a 
home.  Where  was  the  mechanism  to  talk 
about  community  involvement,  or  the 
changes  in  empowerment,  or  the  will- 
ingness to  believe  you  can  make  a 
change  as  a measurement  of  success  for 
programs?  These  measures  do  not  fit 
neatly  onto  a linear  chart  that  denotes 
increase  in  something.  For  most  of  these 
programs,  linear  charts  contained  the 
information  they  believed  the  Common- 
wealth wanted  and  that  contradicted 
what  they  believed  was  their  mission.  At 
this  meeting  the  "Looking  Back  Group" 
decided  to  tackle  the  task  of  discussing 
the  larger  community  changes  within 
their  category  and  create  a fourth  group. 
(This  discussion  is  reflected  in  the 
articles.) 

For  the  next  several  months,  people 
met  in  their  smaller  groups  and  at- 
tempted to  visit  each  other's  programs. 
In  early  October,  we  regrouped,  gave 
updates  and  began  the  topic  of  Quality 
Indicators,  sharing  a copy  of  the  Federal 
indicators  and  a copy  of  the  indicators 
developed  at  ACBE  by  Sondra  Stein  and 
Susan  Rosenblum.  We  decided  to  read 
all  the  articles  and  return  in  November 
to  put  together  recommendations  from 
the  Massachusetts  quality  indicators. 

At  the  November  meeting,  we 
struggled  with  what  we  saw  as  the  real 
issue,  namely  that  the  concept  of  ac- 
countability needed  to  be  changed  from 
a hierarchical  linear  concept  to  a more 


cyclical  concept.  One  suggestion  was 
that  programs  needed  to  be  better  ap- 
prised of  the  needs  of  the  State  DOE  and 
how  they  were  accountable  to  the 
Federal  DOE  so  they  understood  the 
Bureau's  constraints.  Although  the  group 
felt  it  was  difficult  to  improve  the  ACBE 
job,  the  teachers  did  establish  a list  of 
recommendations  and  also  generated 
proposals  for  an  accountability  system 
that  would  reflect  those  quality  indicators. 

After  our  November  meeting,  there 
was  a long  break  until  February.  We 
decided  to  finish  the  "Notebooks"  and  to 
create  documentation  about  the  note- 
books for  others  in  the  field,  and  to 
document  some  changes  that  individuals 
have  gone  through.  It  was  at  this  point 
of  the  process  that  I went  on  to  a new 
job,  leaving  the  group  to  write  up  their 
experiences  and  their  feelings. 

THE  FRUITS  OF  OUR  LABOR 
This  work  was  used  to  develop  the 
indicators  for  the  Commonwealth  of 
Massachusetts.  These  teachers'  articles 
and  the  Alternative  Assessment  Note- 
books reveal  all  the  earmarks  of  quality 
staff  development.  The  process: 

• Networked  teachers  together  across 
disciplines,  programs,  and  the  state; 

• Provided  a chance  to  share  materials, 
methodology  and  ideas  for  further 
programming; 

• Gave  an  opportunity  to  discuss  issues 
at  a time  in  our  profession  when  we 
rarely  have  time  to  stop  and  reflect 
upon  any  form  of  assessment; 

• Enabled  them  to  read  other  people's 
work  (including  but  not  limited  to 
Lytle,  Fingeret,  and  Stein),  and 

• Allowed  them  to  begin  to  document 
their  ideas. 


Where  was  the 
mechanism  to  talk 
about  community 
involvement,  or 
the  changes  in 
empowerment, 
or  the  willingness 
to  believe  you  can 
make  a change  as 
a measurement 
of  success  for 
programs? 
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Even  though  the 
original  intention 
may  not  have  been 
to  create  a 
successful  staff 
development 
experience,  it  was  a 
natural  by-product 
of  die  experience. 


Much  attention  has  been  given  to  the 
use  of  teacher  inquiry  groups  as  good 
staff  development  Lytle  and  Cochran- 
Smith  argue  it  is  a "way  to  generate  both 
local  knowledge'  and  'public  knowl- 
edge' about  teaching;  that  is,  knowledge 
developed  and  used  by  teachers  for 
themselves  and  their  immediate  commu- 
nities, as  well  as  knowledge  useful  to  a 
larger  community"  (1992).  So,  even 
though  the  original  intention  may  not 
have  been  to  create  a successful  staff 
development  experience,  it  was  a natural 
by-product  of  the  experience. 

Because  of  the  different  perspectives 
of  the  teachers,  it  was  similar  to  a multi- 
level classroom.  However,  all  of  the 
teachers  had  years  of  experience  from 
which  to  draw.  Sometimes  it  was  diffi- 
cult to  balance  the  task  of  looking  at  the 
issue  of  accountability  and  what  the 
Bureau  had  hired  me  to  do  and  to 
engage  in  this  staff  development  activ- 
ity. 

As  I have  said  previously,  I found 
this  experience  to  be  very  similar  to  the 
process  of  inquiry  and  to  the  process  of 
staff  development  The  clearest  proof  is 
that  the  practitioners  who  came  into  this 
component  in  April  of  1992  claiming 
they  had  no  tools  in  place,  but  were 
really  interested  in  learning,  are  the  ones 
with  the  most  energy  to  see  this  group 
continue.  They  are  the  ones  committing 
themselves  for  another  year.  Those 
practitioners  who  had  already  given  the 
topic  a lot  of  thought  and  creation  seem 
to  be  interested  in  moving  to  another 


topic  It  is  the  sign  of  good  staff  develop- 
ment when  there  is  excitement 
remaining  in  those  who  still  feel  they 
have  something  to  gain  from  the  experi- 
ence. 

As  for  an  accountability  system,  I 
believe  the  point  of  all  this  work  is  to 
show  that  accountability  cannot  rest  on 
the  learner's  achievements.  Comparing 
learners  does  not  make  a program 
accountable.  The  idea  is  to  work  with  the 
program,  inclusive  of  the  learner,  to 
create  quality  programs.  Those  indica- 
tors of  quality  programs  that  are  agreed 
upon  should  be  measured  by  program 
development  and  not  by  looking  at 
standardized  reading  scores.  Compo- 
nent #3  has  helped  to  better  define  the 
roles  of  assessment  evaluation  and 
accountability,  and  it  is  important  for  us 
to  stop  using  those  words  synonymous 
and  interchangeable.  We  must  be  ac- 
countable to  our  own  knowledge  and 
not  give  in  for  the  sake  of  convenience 


Footnotes 


1 Public  policy  is  usually  created  by  administrative  and 
governmental  offices  and  then  individuals  are  asked  to 
be  responsible  and  implement  that  policy.  This  project 
is  attempting  to  get  around  making  that  mistake  and 
systematically  and  intentionally  asked  for  individual 
and  group  input 

2Intemational  Reading  Association.  (1 988).  Resolution 
on  reading  assessment  Newark,  DE:  Author. 


3 A method  of  training  which  causes  participants  to 
first  define  for  themselves  a concept  and  then,  by 
joining  with  more  and  more  people,  have  to  agree  upon 
a definition  with  a group. 


4 Examples  of  the  types  of  tools  can  be  found  in  Adven- 
tures in  Assessment  Volume  3,  a rtide  by  Eileen  Barry. 
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•This  is  the  original  version  of  the  toolkit.  The  latest  version  was  not  available  at  time  of  publication. 
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Assessment  Practices  & Tools  at  Read/Write/Now 

A Reflection  on  the  Ideal  vs.  the  Real 


by 

Janet  Kelly 


Read/Write/Now 

Springfield 


"Reflection"  may  be  too  serene  and 
contemplative  a word  for  what  I'm 
going  to  do  in  this  piece  and  definitely 
for  the  place  I'm  doing  it  from.  There  is 
traffic  whizzing  by  outside  my  sunporch 
as  I write,  cats  walking  over  my  paper, 
and  the  constant  awareness  of  my  baby 
son's  sleeping  presence  in  the  next  room. 
Is  he  really  sleeping?  Did  that  last  truck's 
passing  wake  him? 

I'm  struck  by  how  like  my  program's 
one  room  environment  this  distraction- 
packed  home  space  is,  at  least  when 
there  are  no  classes  in  action.  In  the 
program  space  the  distractions  are 
endless  phone  calls,  visitors,  various 
staff  people  coming  and  going,  learners 
coming  to  visit  or  check  out  books,  daily 
details,  and  unforeseen  crises  that  are 
all  part  of  life  in  a learning  center. 

Yet,  despite  the  distractions, 
reflection  and  review  of  assessment 
practices  and  tools  still  happen.  They 
have  become  a part  of  what  we  see  as 
necessary  to  make  our  curriculum  as 
responsive  as  possible  to  learners'  goals 
and  needs.  I would  like  to  be  able  to  say 
they  had  a more  central,  deliberate  place 
in  the  work  of  our  program,  but  because 
of  our  small  budget  and  resulting 
shortage  of  staff  hours,  much  of  the 
work  of  assessment  is  fit  into  too  little 
time  around  the  edges  of  classes  and 
preparation  of  materials.  In  principle  it  is 
central,  in  practice  it  doesn't  get  enough 
time  yet. 

We  are  working  on  it.  We  have  a one- 


year  writing  and  publishing  grant  that 
allows  teachers  some  more  hours.  At 
least  this  year  we  have  the  possibility 
of  making  the  real  more  like  the  ideal  in 
terms  of  better  integrating  assessment 
into  curriculum  and  program 
development 

Another  goal  for  this  year  is  to  make 
our  assessment  process  and  tools  more 
responsive  to  the  needs  of  learners.  We 
are  beginning  to  hear  what  learners  have 
to  tell  us  about  the  kinds  of  assessment 
tools  we  use  and  the  way  we  use  them. 

I think  we  have  relied  more  on  what  we 
as  teachers  thought  was  important  to 
learners,  what  we  were  interested  in 
knowing  about  their  progress,  what  we 
thought  we  could  document,  and  what 
we  thought  funders  would  want  to 
know.  These  beliefs  still  form  the  basis  of 
our  choices  in  assessment  practices,  but  I 
hope  we  will  find  ways  to  invite  learners 
to  share  more  effectively  in  decisions 
not  only  about  program  and  curriculum 
evaluation,  which  they  currently  do, 
but  also  in  the  selection  of  and  critique 
of  materials  for  their  own  assessment 
portfolios. 

RETHINKING  TOOLS 
AND  PRACTICES 

After  the  first  two  volumes  of 
Adventures  in  Assessment  came  out,  I 
began  to  rethink  some  of  the  choices  of 
tools  and  practices  I'd  included  with  the 
articles  about  Read/Write/Now.  That's 
partly  because  of  the  nature  of  this  kind 
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of  assessment  as  well  as  the  nature  of 
writing  and  publishing.  Obviously, 
nothing  that's  based  on  human  beings' 
changing  needs  and  goals  is  ever 
finished  or  perfect.  Our  needs  and  goals 
are  continually  changing  and  new 
information  or  a new  perspective  on  an 
old  practice  requires  making  revisions  in 
what  was  "perfect"  for  last  year,  or 
yesterday.  Many  forms  designed  to  fit  a 
perceived  need  in  assessment  have 
seemed  perfect  until  they  were  put  to 
practical  use.  Then  the  revisions  needed 
become  obvious.  Also,  sharing  or 
publishing  anything  I've  put  in  writing, 
whether  it's  a grant  proposal  or  a poem, 
is  one  sure-fire  way  of  seeing  it  with  new 
eyes,  sometimes  through  the  responses 
of  readers  and  sometimes  through  being 
able  to  stand  back  far  enough  to  see  it  in 
a new  way.  This  is  one  of  the  great 
things  about  this  journal  — learning 
from  what  other  people  are  doing  by 
reading  articles  and  talking  to  people 
and  learning  more  about  what  we  are 
doing  by  writing  about  it. 

It's  also  been  interesting  to  hear  about 
ways  other  programs  have  adapted 
things  we  vise;  this  has  helped  us  see  our 
own  practices  in  a new  light,  too.  The 
journal  has  helped  expand  our 
collaborative  style  of  working  together 
to  include  other  teachers  in  other 
programs.  Our  own  program  has 
expanded  to  include  a family  literacy 
program  funded  by  Even  Start,  which  is 
a collaboration  with  the  Springfield 
School  Department  The  Even  Start  staff 
have  brought  new  ideas  and  a family 
literacy  perspective  to  portfolio 
assessment,  which  has  been  a positive 
addition  to  our  overall  assessment 
process. 


WHATS  IN  A R/W/N 
PORTFOLIO  NOW? 

GOAL  SETTING: 

initial  screening  & placement  interview, 
including  responses  to  teacher-made 
reading  comprehension  exercise,  various 
leveled  reading  samples  if  needed,  and 
Slosson  score 
goals  checklist 
learning  contract 

READING: 

list  of  books  read 

riyiforonro  record 

reading  conference  checklist 
reading  progress  checklist 
beginning  readers  progress  checklist 
samples  of  beginning  readers  language 
experience  stories 
reading  miscue  analysis  results 
reading  response  journals,  student 
Individual  reading  records,  self-evaluation 
of  reading  responses 

WRITING: 

first  writing  sample 
dialogue  journal  samples 
other  dated  writing  samples 
writing  conference  record 
writing  progress  checklist 
published  writings 

OTHER: 

learner’s  self-evaluation  forms 

math  activities  record 

computer  activities  logs  - various  programs 

teacher*  logs  (anecdotes  & observations  - 

kept  in  separate  notebook) 

evidence  of  leaming/progress  significant  to 

learners  - copy  of  driver’s  license  earned, 

letters  sent  to  newspapers, 

representatives,  etc 

Janet  Kelly ; Read/Write/Now 
9120193 
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The  Read/Write/ 
Now  Screening 
& Placement 
Interview  as 
published  in 
Adventures  is 
an  example  of  a 
tool  that  seemed 
to  address  a 
perceived  need, 
but  didn't  work  in 
the  real  situation 
at  all. 


CHANGES  IN  THE  READ/WRITE/ 
NOW  PORTFOLIO 

What's  in  a portfolio  at  Read/Write/ 
Now  these  days  and  how  it  differs  from 
what  was  in  a portfolio  two  years  ago  is 
something  I'd  like  think  about  on  paper. 
Looking  at  the  summary  that  was 
included  in  the  November  '91  issue  of 
Adventures.  I see  things  we've  kept 
doing  and  things  we've  let  go  due  to 
lack  or  relevance  or  time. 

In  the  category  of  "Goal  Setting",  we 
continue  to  use  an  extensive  interview  in 
our  initial  screening  with  a placement 
"test"  that  includes  a teacher-made 
reading  comprehension  sample,  the 
Slosson  Oral  Reading  Test  to  evaluate 
word  recognition  and  decoding  skills, 
and  a writing  sample.  The  Even  Start 
adult  education  staff  used  the  Fry 
readability  scale  to  level  various 
passages  of  writing  by  adult  learners 
and  created  questions  to  assess 
comprehension  for  more  definitive 
placement  of  learners. 

The  Read/Write/Now  Screening  & 
Placement  Interview  as  published  in  VoL 
1 of  Adventures  is  an  example  of  a tool 
that  seemed  to  address  a perceived  need, 
but  didn't  work  in  the  real  situation  at 
all.  On  the  final  page  I had  put  in  a 
statement/ question  about  drugs  and 
alcohol  use  that  I thought  would  clarify 
a previously  unstated  and  murky  policy. 
It  was  to  be  read  and  signed  by  both 
interviewers  and  interviewees.  The  first 
time  I tried  to  use  it  I found  that  I 
couldn't;  the  first  face-to-face  meeting 
with  a prospective  learner  didn't  feel 
right  for  dealing  with  this  issue.  Things 
sometimes  look  so  appealing  on  paper 
but  make  no  sense  in  practice.  So  that 
part  of  the  form  was  eliminated. 


Our  Goals  Checklist  has  stayed  about 
the  same  for  a couple  of  years,  with  only 
minor  revisions.  The  Even  Start  team 
used  it  as  a basis  for  designing  a family 
literacy  goals  checklist,  and  added  a 
feature  of  choosing  short  and  long  term 
goals.  Learning  Contracts  are  still  vital 
pieces  of  our  program  and  assessment 
process.  We  have  seen  a lot  of  growth  in 
both  learners'  and  teachers'  ability  to  use 
this  tool  to  name  and  reach  goals.  We've 
all  gotten  better  at  being  more  realistic 
and  organized  about  arriving  at 
workable  goals  and  breaking  them  into 
understandable  steps.  New  staff  are 
helped  in  this  by  seasoned  learners  and 
staff  and  new  learners  are  helped  in  the 
same  way.  Knowing  our  limits  without 
limiting  ourselves  too  much  is  one  of  the 
great  balancing  acts  of  adult  life,  it 
seems.  We're  always  working  on  that 
one. 

One  change  we've  made  in  relation  to 
Learning  Contracts  is  setting  aside  time 
for  individual  learning  contract 
conferences  at  the  beginning  of  each 
class  cycle  instead  of  trying  to  do 
lengthy  conferences  during  regular  class 
time.  This  has  helped  us  to  really  hear 
each  person,  find  out  what  they  need 
and  want  to  do,  and  also  helps 
determine  how  committed  a learner  is 
who  we  have  questions  about  from  past 
history.  Sometimes  a contract  review 
session  includes  an  update  and  slight 
revision  of  a learner's  goals  rather  than  a 
whole  new  set  of  goals. 

Often,  the  review  shows  a 
progression  from  one  set  of  goals  to  a 
logical  next  set.  James  M.,  who  accomp- 
lished his  long-term  goal  of  attaining  a 
Commercial  Driver's  License  after  his  20 
year  factory  job  ended  when  the  factory 
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dosed,  got  a job  driving  a bus  this 
summer.  His  goals  for  this  foil  indude 
improving  his  map  reading  skills. 

Some  of  the  tools  for  reading  and 
writing  assessment  are  still  used  as 
described  in  Vols.  1 & 2 of  Adventures 
while  others  have  faded  away  from 
disuse,  been  purposefully  retired,  or  are 
used  differently  now.  We  still  use 
learners'  book  lists  and  reading 
conference  records  as  tools  for  ongoing 
assessment.  In  addition,  Marilyn 
Antonuca,  a Read/Write/Now  teacher 
who  is  now  a part  of  the  Even  Start 
team,  developed  anoiher  form,  the 
Reading  Conference  Checklist,  which 
provides  a framework  for  seasoned  staff 
to  do  a more  comprehensive  reading 
conference  or  helps  new  staff  and 
substitutes  understand  what's  involved 
in  an  individual  reading  conference.  This 
form  is  used  as  needed  throughout 
cycles. 

The  Reading  Progress  Checklist  is 
still  used  to  record  development  of 
reading  strategies  of  all  but  beginning 
level  readers.  The  comments  section  has 
become  a place  to  summarize  reading 
progress  as  observed  by  teachers.  A 
Beginning  Readers  Progress  Checklist 
was  developed  by  Marilyn  Antonucci  to 
assess  the  development  of  reading 
behaviors  and  strategies  by  learners  in 
the  "0  - 1.9"  reading  level  category. 

These  learners'  portfolios  also  indude 
samples  of  their  language  experience 
stories  and  doze  activities.  Two  tools 
that  have  faded  away  are  "Looking  At 
Your  Own  Reading  Behavior"  and  the 
modified  Burke  Reading  Interview. 

They  may  be  used  in  reading  groups 
as  a stimulus  for  discussion  about  the 
reading  process,  especially  in  a group  of 


learners  new  to  the  program. 

Reading  Miscue  Analysis  continues 
to  be  an  important  component  of  our 
reading  assessment,  but  routine  detailed 
miscue  analysis  with  all  learners  for 
whom  it  is  appropriate  is  still  a goal 
rather  than  a reality.  The  biggest  barrier 
to  frequent,  comprehensive  miscue 
analysis  has  been  too  few  staff  to 
schedule  the  uninterrupted  time  needed 
with  learners  to  do  the  miscue  inventory 
and  analysis  afterwards.  We  also  need 
more  staff  development  time  to  practice 
doing  miscues  so  that  more  than  one  or 
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Reading  Response  Journals  have  also 
been  added  to  some  learners'  portfolios. 
Readers  who  have  gotten  involved  in 
writing  responses  to  their  independent 
reading  have  chosen  to  use  another  one- 
page  form  to  periodically  evaluate  their 
own  written  response.  This  encourages 
the  development  of  critical  reflection  of 
reading  and  validates  the  active  aspect 
of  the  reading  process  — what  the 
reader  brings  to  the  text  and  what  . 
meaning  he/she  makes  of  it 

Writing  assessment  still  consists  of  a 
first  writing  sample,  dialogue  journal 
samples,  other  dated  writing  samples 
and  published  writings.  We  have  aimed 
for,  but  not  completely  routinized, 
learners  and  teachers  deciding  together 
about  what  goes  into  portfolios.  We 
need  to  add  a simple  statement  about 
who  picked  which  piece  and  why.  We 
know  this  at  the  time  and  think  we're 
going  to  remember,  but  sometimes  it 
gets  lost  along  the  way.  The  Writing 
Progress  Checklist  has  been  substan- 
tially revised  to  reflect  the  kind  of 
analysis  of  learners'  writing  we  really 
do  and  have  time  for.  The  form  is  used 
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twice  in  a cycle  and  compared.  The 
Writing  Conference  Record  kept  in  each 
learner's  writing  folder  has  been  revised 
slightly  to  encourage  recording  any 
writing  activity,  rather  than  just  "skills 
worked  on".  Several  teachers  have  found 
it  useful  to  keep  more  involved  logs  of 
where  learners  are  in  their  writing,  but 
these  notes  don't  find  their  way  into  the 
portfolios. 

LEARNING  LOGS  A FAVORITE,  BUT 
DISCARDED  FOR  NOW 

A poet/teacher  once  said  his  favorite 
line  in  a poem  is  the  one  that  needed  to 
be  thrown  out  to  make  the  poem  work. 
Learning  logs  were  like  that  favorite  line 
for  me.  I was  very  attached  to  the  idea, 
partly  because  I did  something  like  them 
in  an  alternative  education  class  in  high 
school  and  loved  it  However,  nobody 
else  liked  them,  learners  or  staff,  so  they 
were  thrown  out.  I keep  hoping  to 
devise  some  variation  on  the  logs  that 
will  mean  more  to  people.  The  format 
used  the  same  four  statements  to  elicit 
learners'  comments  on  their  own 
learning  in  and  out  of  the  program  on  a 
weekly  basis.  Many  learners  gave  rote 
answers,  often  using  the  previous  week's 
log  as  a pattern.  Most  people  felt  it  was 
something  they  were  doing  for  teachers, 
not  for  themselves.  Although  some 
people  did  start  to  use  the  logs  to  reflect 
on  their  learning  and  let  us  know  about 
things  they  liked  and  didn't  like,  most 
people  never  seemed  to  feel  ownership 
of  the  logs.  We've  considered  using  one 
different  question  each  week,  but  we 
haven't  tried  it  yet  Meantime  there  is 
always  a lot  of  talk  about  learning,  about 
what  works  and  doesn't,  but  learners  are 
not  necessarily  putting  it  into  writing. 


This  kind  of  folk  often  finds  its  way  into 
dialogue  journals. 

We  continue  to  use  a revised  form  of 
the  Teacher's  Log  for  Observations  & 
Reflections  and  find  it  very  helpful.  One 
challenge  is  finding  a way  to  make 
significant  entries  in  the  logs  part  of 
learners'  portfolios.  The  logs  are 
primarily  used  for  teachers  to  reflect  on 
how  particular  learners  are  doing, 
document  evidence  of  literacy  learning 
and  growth  in  self-esteem,  and  make 
note  of  interests  and  needs  to  follow  up 
on.  Significant  anecdotes  and 
observations  could  be  recorded  on  a 
joint  teacher/leamer  form  each  learning 
contract  period  or  month.  This  would 
give  definite  shape  to  teachers  sharing 
their  observations  with  learners  and  act 
as  another  forum  for  learners  to  tell 
teachers  what  they  have  observed  about 
their  own  learning.  ( I hear  the  Read/ 
Write/Now  staff  saying:  "Somebody 
stop  her  before  she  creates  another 
form!") 

Learners'  self-evaluation  forms, 
which  include  some  program  evaluation, 
are  revised  as  needed  by  staff  every 
cycle  It  would  probably  be  very  useful 
to  ask  learners  to  help  us  devise  a better 
way  to  elicit  critical  responses  to 
questions  about  program  design  and 
effectiveness.  What  seems  to  work  best  is 
time  and  trust  The  longer  learners  are 
part  of  the  learning  center  community, 
the  better  they  get  at  telling  us  what 
works  for  them,  what  doesn't,  what  they 
like  and  don't  like  and  why. 

SOME  TIME  TO  THINK,  PLEASE 

If  I won  the  lottery  and  could  start  an 
endowment  fund  for  Read/Write/Now 
so  that  shortages  did  not  exist,  one  thing 
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I'd  do  (after  making  all  staff  full  time 
with  benefits,  adding  more  classes,  and 
having  windows  put  in  our  room)  is  hire 
a clerical  person  to  help  us  maintain  the 
organization.  This  mythical  person  could 
also  answer  the  phone  and  do  other 
essential  work  that  would  allow  teachers 
and  myself  to  spend  more  time  on 
making  the  real  assessment  process  and 
tools  a!  little  more  like  the  ideal  ones  we 
keep  striving  for. 

One  thing  I have  recognized  in  these 
few  years  of  being  involved  in 
alternative  assessment  is  that  we  can't  go 


back.  Standardised  tests  alone  would 


never  capture  the  range  and  richness  of 
literacy  learning,  personal  and  social 
growth,  awareness  and  involvement  in 
community  life  and  issues,  and  that 
overused  but  apt  work,  'empowerment' 
that  happens  in  the  lives  of  adult 
learners.  Tests  alone  could  not  show 
these  things  in  ways  authentic  enough  to 
satisfy  either  learners'  or  teachers'  need 
to  know.  So,  lottery  or  no  lottery,  we  are 
committed  to  this  process  of  continual 
rethinking,  reflecting,  streamlining, 
expanding,  and  revising  assessment 
tools  and  the  ways  we  use  them. 

5T 
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READING  CONFERENCE  CHECKLIST 

WHAT*  teacher's  checklist  to  organize  information  from  an  individual  reading  conference  with 
developing  and  more  advanced  readers 

WHY*  helps  teacher  categorize  and  analyze  information  from  conferences,  as  well  as  remind  her/ 
him  which  areas  to  pay  attention  to  in  reading  conferences 

HOWs  on  an  individual  basis,  as  needed 

WHEN*  especially  helpful  when  a teacher  is  new  to  reading  conferences;  done  throughout  the 
class  cycle,  although  not  necessarily  with  every  reading  conference. 


Name  of  Student 
Name  of  book  


Date 


Approximate  level 


How  well  did  student  enjoy  this  book  Very  Well  Some 
Appropriateness  of  this  selection:  Good 


Not  Very  Well 
Too  Easy  Too  Hard 


I.  Comprehension:  General  Understanding  of  book  □ Good  □ Fair  □ Poor 


I.  Retelling 

□ Unaided  recall 

□ Accuracy  of  recall 

□ Recall  when  asked  questions 

1 Recognition  of 

□ Character  identification 

□ Character  descriptions 

□ Events  in  succession 

□ Plot  summary 

II.  Oral  Reading:  Fluency: 

Rate:  □ Good  □ Too  Slow  I 

□ Word-by-word  reading 

□ Poor  phrasing 

□ Lacks  good  sight  vocabulary 

□ Reversals  letter/word/phrase 

□ Gross  mispronunciations 

□ Substitutions 

□ Self  corrects 

□ Responds  to  unknown  words  by: 


Very  Good 


Good 


Fair 


Poor 


Too  Fast 

□ Ignores  punctuations 

□ Repetitions:  word/phrases 

□ Omissions  sounds/words 

□ Additions:  sounds/words 

□ Confuses  similar-looking  words 

□ Loses  place  frequently 


Fair 


Poor 


III.  Word  Recognition:  General  accuracy  of  word  perception:  Good 
Needs  help  in 

□ Context  dues 

Syntactic  context  dues  signals  provided  by  word  endings,  function  words  and  word  order 
Semantic  context  dues  meaningful  relations  among  words 

□ Configuration  dues  the  shape  of  the  word 

□ Phonemic  analysis  knowledge  of  wrods  or  word  parts  (eg.  window  and  sill=windowsiII) 

□ Structural  analysis  knowledge  of  affixes  and  bases  of  words 

□ Accurate,  rapid  word  recognition 
Summary: 

Recommendations 
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SELF  EVALUATION  OF  READING  RESPONSES 

WHAT:  learner's  form  to  use  to  reflect  on  own  reading  responses  in  response  journals 

WHY:  gives  more  advanced  readers  an  opportunity  to  look  back  at  and  value  the  growth  and 
variety  of  their  responses  to  reading  they  have  had  over  a period  of  time 

HOW:  learner  looks  over  response  journal  of  a period  of  time  (6-8  weeks)  and  then  uses  the  form 
to  reflect  critically  on  the  responses  and  the  process 

WHEN:  intermittently,  throughout  the  class  cycle. 

A response  journal  is  a notebook  or  folder  in  which  students  record  their  own  personal  reactions  to, 
questions  about,  and  reflections  on  a book  they  are  reading  with  a group  or  independently.  A 
response  can  also  be  about  a TV  show,  a movie,  a meeting,  or  a family  event,  that  was  meaningful  to 
the  student. 

The  student  is  developing  awareness  of,  and  eventually  commitment  to,  their  own  learning  pro- 
cesses necessary  to  help  them  develop  effective  reading  strategies. 

The  Response  journal  is  read  by  the  teacher  and,  because  ii  is  "personal"  writing,  it  is  net  marked 
for  mechanical  acuracy  or  stylistic  features.  A reply  to  the  student  is  not  necessary  in  the  journal 
itself. 


Name  of  Student 


Date 


Evaluation  period  from 


to 


1 . With  which  response  are  you  most  satisfied? 
Why? 


1 Wfth  which  response  are  you  most  dissatisfied? 
Why? 


3.  What  can  you  do  to  try  to  make  your  responses  more  satisfying? 
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BEGINNING  READER  PROGRESS  CHECKLIST 

WHATs  teacher's  comprehensive  checklist  for  assessment  of  individual  beginning  reader's  reading 
behavior  and  development  over  a period  of  class  cycle 

WHYl  to  organize  and  categorize  reading  assessment  information  in  an  accessible  format  to  help 
teachers  plan  activities  for  individuals  and  reading  groups  as  well  as  to  document  learner's 
progress  in  the  beginning  stages  of  reading  development 

HOWs  teacher  looks  over  logs  and  reflects  on  learner's  reading  behavior  in  class,  individual  and 
group  work  with  reader,  learner's  self-assessment  and  reporting  of  reading  behavior  outside  of 
class  and  complete  checklist 

WHENs  towards  end  of  learning  cycles 

(adapted  and  expanded  by  Janet  Kelly  from  Sylvia  Greene's  Writing  & Spelling  Progress  Sheet,  from  "Basic 
Literacy  Kit"  (1989) 

Reader  Teacher  Date: 

Consistency  Sometimes  Not  Yet 

Evident  Evident  Evident 

I:  Gaining  confidence  in  self  as  potential  readers 

1 . Follows  the  sequence  of  events  in  a story 

2.  Expects  words  to  make  sense 

3.  Uses  directional  conventions 

3a  Follows  print  in  left  to  right  direction 
3b.  Goes  to  next  line  (left  again  on  line  below) 

4.  Understands  that  for  every  spoken  word,  there  is  a 
written  one  (one-to-one  correspondence) 

5.  Understands  that  the  shape  of  a written  word  remains 
constant 

6.  Learning  to  use  the  following  cues: 

6a.  Picture  cues 

6b.  Semantic  cues  (what  makes  sense) 

6c  Syntactic  cues  (what  sounds  Bee  language) 

Expects  text  to  follow  accepted  speech  patterns 
6d  Graphophonic  cues 
6di.  Developing  a sight  vocabulary 
6d2  Reading  words  form  own  vocabulary 
6d3.  Reading  words  from  reading  materials  (LEA  and 
reading  texts) 

Consistently  Sometimes  Not  Yet 
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Consistently  Sometimes  Not  Yet 

Evident  Evident  Evident 

7.  Developing  concepts  about  print 
7a  Identifies  alphabet  letters 

7b.  Prints  letters  (manuscripts  form) 

7c  Writes  letters  (cursive  form) 

7d  Recognizes  punctuation  marks: 

7d I.  period 
7d2  comma 
7d3.  question  mark 
7d4.  exclamation  mark 
7d5  quotation  marks 

7e  Understands  spacing  between  words  / 

7f.  Understands  one-to-one  word  matching 
7g.  Building  awareness  of  shapes  of  words 
7h.  Building  awareness  of  length  of  words 

8.  letter/sound  responses  (identifies  sounds): 

8a  Initial  consonants 

8b.  Consonant  clusters 

8c  Short  vowd  sounds 

8d.  Long  vowel  sounds 

8e  Double  vowel  sounds 

8f.  Makes  simple  substitution  of  initial  consonants 

9.  Word  analysis 

9a  Sees  the  endings  on  words,  -s,  ed,  ing 
9b.  Sees  compound  words 

10.  Developing  a pattern  of  strategies  for  getting 
meaning  from  words 

1 0a  Predicts  which  word  makes  sense 
I Ob.  Understands  oral  doze  activities 
1 0c  Confirms  if  word  does  make  sense 
!0d.  Initiates  a new  word  which  does  make  sense 
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Consistently  Sometimes  Not  Yet 

Evident  Evident  Evident 


II.  Involvement  In  Writing 

1 . Participates  in  Shared  Reading 

I a Reads  daily  news  story  (teacherfleamer  generated) 

I b.  Reads  text  of  stories  from  Big  Books  (teacher  made 
materials/commerrial) 

2.  Developing  reading  skills  by  Shared  Reading  (supported 
reading) 

2a  Enjoys  reading  with  others 
2b.  Understands  about  directional  conventions 
2c  Reads  familiar  refrains  in  a shared  reading  group 
2d.  Reads  the  simpler  words  in  a shared  reading  group 
2e.  Increasing  listening  and  spoken  vocabulary  through 
story  language 

2f.  Increasing  own  understanding  of  more  complicated 
sentence  structure 

2g,  Uses  new  sentence  patterns  in  conversations 

2h.  Developing  ear  for  more  varied  vocabulary 

21  Increasing  comprehension  by  understanding  more 

complicated  storyline 

2j.  Increasing  sight  word  bank 

2k.  Developing  fluency 

21  Developing  intonation,  appropriate  voice  tones 
2m  Rereads  a shred  readng  text  independently 
2a  Shares  in  discussion  about  story  content  with  group 

3.  Developing  as  a reader  by  using  other  supported  reading 
activities 

3a  Listens  to  tapes 

3b.  Listens  to  tapesftums  the  pages/looks  at  ^lustrations 
3c  Follows  the  words  with  finger  whBe  listening  with  tapes 
3d.  Selects  and  reads  an  appropriate  text  with  assistance 
(assisted  reading) 

3e.  Choral  reads  with  a peer  (paired  reacfcng) 
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Consistently  Sometimes  Not  Yet 

Evident  Evident  Evident 

4.  Individualized  reading 

4a.  Reads  own  writing  materials 

4b.  Reads  familiar  bools 

4c  Choosers  new  bods  to  read  independently/taking  new 
bods  home. 
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A study  of  the  practice  of  alternative  assessment 

DANGER:  Road  Construction  Ahead 


by 

Paul  Trunnell 


Harboiside  Community 
Center 
East  Boston 


When  A.  first  joined  the  class  one  year  ago, 
she  met  with  L.,  the  previous  teacher.  The 
classroom  was  small  with  fluorescent 
lighting,  but  sun  shone  through  the  win- 
dowpanes.  The  walls,  painted  yellow,  were 
concrete  blocks.  Other  students  worked  on 
their  assignments  quietly,  alone  or  in 
groups,  and  the  assistant  teacher  moved 
among  them.  In  the  next  room  children's 
voices  rang  out  amidst  the  banging  of  blocks 
and  toys. 

L.  and  A.  sat  down  at  a table  to  discuss 
A/s  reasons  for  coming  to  school.  A.  spoke 
quietly  to  her  new  teacher,  conscious  of  her 
accent  and  carefully  choosing  her  words.  She 
was  searching  for  response  in  her  new 
teacher  and  considering  her  new  environ- 
ment. L.  spoke  softly  too,  making  a lot  of  eye 
contact  and  trying  to  make  A.  feel  comfort- 
able. After  talking  for  a few  minutes,  L. 
suggested  they  do  a formal  oral  language 
assessment  as  a way  to  better  understand 
A.'s  needs.  Afterwards,  L.  went  over  the 
results  of  the  assessment.  A.  began  to  speak 
more  freely  about  her  difficulties  with 
language  and  what  she  hoped  to  accomplish. 
L.  listened  carefully  and  made  suggestions 
about  activities  to  pursue.  At  the  end  of  the 
meeting  L.  gave  A.  a book  to  start  working 
on.  A.  smiled  and  thanked  her  new  teacher. 

After  class,  L.  wrote  up  this  initial 
interview: 

"Speaks  very  well.  There  are  some 
pronunciation  difficulties  which  are  being 
corrected  with  hearing  and  reading  the 
word.  A.  has  taught  herself  to  speak  English 
and  does  very  well.  She  is  concerned  that. 


when  faced  with  a crisis  situation  such  as 
son  in  the  hospital  cut  foot  she  can't  express 
herself  in  English  at  all.  We  will  work  on 
this  with  vocab  role  playing. 

"A.  will  try  to  find  the  time  after  and 
during  her  busy  day  caring  for  her  family  to 
practice  writing.  Write  notes  to  her  son  who 
will  respond  with  another  written  note  for 
her  to  read  etc.  She  will  make  shopping  lists, 
lists  of  chores.  Practice  writing  cursive. 
Read  labels.  Read  children's  books  to 
Rosanna  with  practicing  reading  the  story 
ahead  of  time  for  understanding  pronuncia- 
tion, then  to  interject  excitement.  She  will 
get  a pocket  dictionary  of  any  sort  to  use. 

“Practice  math  skills  mult  div. 

"Review  the  folder  of  work  completed  by 
copying  and  reading." 

I reprint  this  episode  in  full  because  I 
believe  it  exemplifies  much  of  what 
alternative  assessment  can  be.  The 
whole  experience  has  been  one  of  get- 
ting to  know  each  other.  The  student  has 
been  thoroughly  assessing  her  new 
surroundings  as  the  teacher  similarly 
assessed  the  student's  needs.  The  com- 
fort of  the  interaction  has  set  the  tone  for 
the  work  to  come.  A.  is  shown  that  her 
needs  are  important  and  that  the  teacher 
is  creating  activities  specific  to  these  needs. 

We  learn  in  the  record  not  only  the 
teacher's  point  of  view  but  also  A/s  own 
concern  about  speaking  in  a crisis 
situation.  The  teacher  has  also  guided  A 
through  an  extensive  list  of  learning 
activities  that  A.  can  do  in  school  or  at 
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home.  All  these  can  be  touched  on  at  a 
later  assessment  date.  Improvement  can 
be  clearly  defined  in  terms  of  personal 
goals,  not  disconnected  skill  attainment. 
Finally  the  narrative  is  translatable  to 
the  next  teacher  walking  in  the  room — 
in  this  case,  me. 

Alternative  Assessment.  I've  read  and 
spoken  about  it  in  very  definitive  and 
appreciative  language.  Why  then  is  the 
practice  so  problematic — while  simulta- 
neously rich? 

At  a day  devoted  to  teacher  inquiry, 
Dulaney  AlsxEncisr  of  Oponition  Boot- 
strap  in  Chelsea  discussed  the  progress 
her  program  has  made  in  the  past  year 
of  alternative  assessment.  To  para- 
phrase, she  said,  "Well,  we  don't  use 
those  forms  anymore.  But  the  philoso- 
phy behind  them  has  become  much 
more  rooted  in  our  practice." 

I remember  her  comment  because  it 
is  true  for  me.  A year  ago  I submitted  an 
article  to  this  journal  in  which  I dis- 
cussed self  assessment  tools  and 
methods  I was  using  in  my  class  and  at 
our  program.  Many  of  the  forms  were 
minimal.  Some  had  only  existed  for  a 
few  months.  Well,  here  it  is,  a year  later, 
and  those  forms  are  nowhere  to  be 
found  in  my  class  or  this  program.  But  I 
am  continuing  my  efforts  to  better 
understand  my  process  as  a teacher  in 
exploring  and  developing  alternative 
assessment.  In  this  article,  I look  at  the 
ways  I have  experimented  and  the 
choices  I have  made  along  the  way. 

When  I sat  down  to  write  this,  I 
realized  I had  a lot  more  questions  than 
answers.  In  fact  I am  going  to  be  reveal- 
ing a lot  of  (gulp)  mistakes,  which  lord 
knows  teachers  don't  really  make. 


I've  been  teaching  for  two  and  a half 
years  at  the  Harborside  Community 
Center  in  East  Boston.  I've  worked 
mostly  with  an  ABE/Pre-GED  popula- 
tion of  diverse  personal  and  ethnic 
backgrounds.  My  academic  focus  has 
generally  included  fractions,  complete 
sentences,  the  food  chain,  the  Conquest 
of  Early  America  and  the  presidential 
race.  I have  worked  in  classes  varying  in 
size  from  five  to  15  students  with  a wide 
range  of  skill  and  ability  levels.  The 
students'  goals  have  similarly  ranged 
from  GED  acquisition  as  a road  to 
increased  errmlovment:  to  increased  self- 

- A * 

esteem;  to  basic  language  improvement 
for  the  sake  of  day  to  day  communica- 
tion. 

In  the  class  I teach  now,  each  student 
has  something  similar  to  a progress 
portfolio.  The  portfolio  includes  initial 
registration  and  assessment  tools: 
writing  samples,  a math  survey,  a 
learning  and  goals  questionnaire,  and  a 
reading  analysis,  all  of  which  are  re- 
corded onto  math  and  grammar 
checklists  and  an  initial  learning  con- 
tract These  documents  are  updated — 
ideally  — about  six  weeks  into  the  cycle 
and  then  again  at  the  end  of  the  cycle.  In 
addition,  students  keep  learning  jour- 
nals in  which  I respond;  I keep  a 
teaching  journal  of  anecdotal  informa- 
tion; and  the  class  participates  in  a 
beginning  planning  session  and  a final 
class  evaluation  that  provide  the  foun- 
dation for  the  curriculum. 

The  learning  contracts  and  evalua- 
tions are  very  dialogue-oriented.  I meet 
with  students  to  discuss  their  progress 
and  plan  their  goals  and  document 
what's  been  said.  To  this  end,  even  the 
writing,  math  and  reading  samples  and 
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These  tools  are 
valuable,  in  some 
way,  to  me,  the 
teacher,  but  not 
necessarily  to  the 
student.  If  you 
asked  them  about 
these  tools,  they 
would  likely  have 
much  different 
responses 


checklists  form  the  conversation-starters, 
as  it  were.  I find  the  meat  of  what  we 
plan  comes  through  the  process. 

Perhaps  a useful  way  to  proceed  is 
to  sit  down  with  A/s  and  M/s  folders, 
looking  through  the  contents,  and  being 
guided  by  the  use  of  the  materials 
within:  learning  contracts,  checklists, 
work  samples,  self-assessments,  and 
teacher  journal  entries.  I have  used  these 
tools  most  consistently  and  they  should 
bear  the  weight  of  analysis. 

I have  chosen  these  students  because 
the  dialogue  between  us  has  been 
extensive  and  clearly  illustrates  to  me, 
and  I hope  for  the  reader,  many  of  the 
uses  and  purposes  I have  in  assessment. 

Though  I relate  the  educational 
activities  of  two  students,  the  focus  is 
not  on  their  learning  or  my  teaching,  but 
on  the  use  and  effectiveness  of  the 
assessment  tools  and  activities.  Also, 
these  tools  are  valuable,  in  some  way,  to 
me,  the  teacher,  but  not  necessarily  to 
the  student  If  you  asked  them  about 
these  tools,  they  would  likely  have  much 
different  responses  (and  potentially 
much  different  assumptions  about  what 
should  be  assessed  and  what  indicators 
they  need). 

STUDENT  A: 

A/s  folder  is  more  incomplete  than 
some.  Her  initial  intake  assessment  is 
missing;  some  learning  questionnaires 
are  not  filled  out  Checklists  stare  out 
like  rock  face  carved  by  some  prehistoric 
gathering,  in  some  incomprehensible 
language.  A.  sees  her  folder  only  once 
every  six  weeks  or  even  fourteen  weeks, 
if  the  middle  of  the  cycle  seems  too 
hectic  to  slow  down  for  a period  of 
assessment 


Nevertheless,  A/s  progress  is  clearly 
revealed  in  the  teacher's  log  and  the 
writing  samples  discussed  below.  The 
other  tools  were  not  as  successful 

LEARNING  CONTRACTS  — The 
Problems.  A/s  folder  contains  only  two 
learning  contracts  for  a one-year  period. 
The  first  is  the  narrative  that  I reprinted 
at  the  beginning  of  this  article.  The 
second  is  dated  April  27  — eight  months 
after  the  first  one. 

How  was  the  first  one  used?  Actu- 
ally, not  at  all  by  me.  I began  fresh  in 
September  when  I started  teaching  the 
class.  I gave  all  the  students  an  incoming 
assessment  and  held  initial  interviews 
with  them.  The  info  in  the  learning 
narrative  is  basically  the  same  info  that  I 
got  from  A.  in  the  falL 

Was  that  info  recorded?  Yes,  on 
notebook  paper  that  was  never  trans- 
ferred to  a learning  contract.  At  that 
time  I had  not  been  using  these  tools  for 
some  time;  they  had  posed  more  prob- 
lems than  solutions:  too  time 
consuming,  not  readily  valuable,  etc. 
Valuable  though  was  the  discussion.  As 
I said,  A.  told  me  all  of  what  was  in  her 
previous  learning  narrative.  This  discus- 
sion, in  a teacher-student  fashion, 
formed  the  basis  of  an  oral  learning 
contract  which  we  talked  about  over  the 
cycle.  Also  A.  was  placed  in  math  and 
reading  activities  based  on  her  initial 
assessment. 

In  December  the  class  conducted  our 
next  major  assessment.  Once  again,  A. 
and  I discussed  the  next  moves  and  I 
recorded  the  interview  on  notebook 
paper  (that  unfortunately  disappeared 
under  the  piles  on  my  desk). 

The  April  learning  contract  (Figure  1) 
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At  this  point  I felt 
I was  coming  to 
understand  the 
purpose  and  use 
of  these  tools 
again.  I wanted  to 
have  some  sort  of 
record  of  our 
discussions. 


is  reprinted  here.  Much  more  structured 
than  the  learning  narrative,  it  specifies 
academic  categories  and  a "personal/ 
career"  category.  Why  did  the  April 
contract  survive?  At  this  point  I felt  I 
was  coming  to  understand  the  purpose 
and  use  of  these  tools  again.  I wanted 
to  have  some  sort  of  record  of  our 
discussions. 

A.  and  I sat  down  with  her  notebooks 
and  discussed  what  had  been  good  and 
what  had  been  difficult  This  contract 
focused  heavily  on  strategies  and  tasks 
that  A.  could  work  on  independently. 
Again,  the  curriculum  itself  would  be 
structured,  as  much  as  possible,  to 
address  the  variety  of  student  needs  in 
the  class.  The  activities  this  time  focused 
heavily  on  reading  (science)  and  writ- 
ing. Math  was  eliminated  because  Anna 
and  other  students  felt  that  they  were 
having  difficulty  focusing  on  too  many 
subjects  at  once. 

A/s  progress  is  noted  only  in  the 
comment,  "Spelling  List  is  Great!" 

Was  this  independent  work  com- 
pleted? No.  Why  not?  A.  never  got  to  it 
In  class  she  continued  writing  and 
reading  science — very  high  level  for  her 
ESL  abilities.  She  and  other  ESL  level 
students  focused  periodically  on  gram- 
mar and  spelling. 

So  this  makes  learning  contracts 
sound  awful  (or  at  least  my  use  of 
them).  Can  I say  anything  in  my  de- 
fense? Yes.  They  were  not  the  only 
source  of  assessment  for  A. 

TEACHER'S  LOG — Success.  Based  on 
our  discussion  in  December,  A.  and  I 
focused  on  her  writing.  Our  process  in 
this  endeavor  is  shown  most  clearly  in 
the  teacher's  log  that  I keep  daily.  Here 


are  the  entries  I penned  for  A.  that 
Spring.  These  show  her  progress  from 
being  a frustrated  writer  to  having  a 
much  better  attitude  and  much  better 
success. 

1 /20  — "Writing  is  daunting." 

1/27  — "Her  writings  were  very 
good,  did  not  question  her  spelling  or 
grammar,  took  her  some  time  to  com- 
plete 11/2  hours." 

2/1  — "...  took  2 1/2  hours  to  do  the 
MLK  writing  assignment  T don't  like  to 
write/" 

2/3  — wrote  a whole  page — got 
started  on  her  own,  we  talked  it  out  two 
or  three  times  and  then  picked  the  very 
first  thing,  then  the  second,  and  so  on. 
Did  very  well.  Worked  on  spelling 
afterwards:  Rules  and  sight  words." 

2/4  — "A.  to  practice  spelling  is 
copying  the  story  Little  Italy — decided 
to  do  it  on  own.  She  enjoys  story  and 
said  that  spelling  words  that  we  went 
over  she  saw  in  the  story.  Making 
connections.  Is  self-motivated." 

2/10  — "Writing,  spelling,  copied, 
seems  more  confident.  Grammar 
scrambles.  Checked  her  own  spelling 
first.  Got  about  half.  Had  Lorenzo  [a 
fellow  student]  help  and  then  I helped. 
Used  her  spelling  list" 

3/10 — "A.  once  again  — Tm  learn- 
ing a lot'" 

4/7  — "Finished  newspaper  stories — 
her  spelling  is  getting  very  good...  she 
also  made  a Tiide-a-word'  for  the  skills 
page..." 

4/13  — "A/s  writing  is  getting 
longer,  more  comfortable.  We  went  over 
past  tense  verbs  in  a story  she  wrote  last 
night" 
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The  log  records  a dear  sense  of 
progress.  This  record  can  be  made 
available  to  the  student  to  show  her  the 
progress  she  has  made. 

Though  some  of  the  entries  are  not 
backed  up  with  details  or  reasons,  many 
entries  explain  the  strategies  and  activi- 
ties that  she  is  using  to  success:  her 
personal  spelling  list,  having  her  peers 
assist  her,  ignoring  grammar  and  spell- 
ing errors  to  encourage  her  work  on 
content,  having  process  conversations 
with  her.  Also  some  comments  cause  me 
to  reflect  on  assignments  that  were  ill- 
fitted  for  Anna,  for  example,  a writing 
on  Martin  Luther  King  Day  when  she 
was  not  fully  aware  of  who  he  was. 

The  entries  have  emerged  when  she 
and  I would  discuss  her  progress,  in  the 
form  of  encouragement — "remember 
when  you...".  In  this  way  it  served  as  a 
reminder  for  me. 

When  A.  looked  at  this  record  (I  gave 
it  to  her  as  I was  preparing  this  artide) 
she  recalled  all  I had  written  and  was 
proud  of  the  progress  she  had  made. 

CHECKLISTS.  Checklists  (Figure  2) 
proved  problematic  with  A.  We  had  a 
grammar  and  a basic  math  skills  check- 
list The  math  checklist  was  too  broad 
and  did  not  represent  the  advances  A. 
was  making  — advances  that  were 
small,  skill-wise,  but  were  very  notice- 
able regarding  strategies.  She  was 
successful  working  on  math  if  she  had 
the  opportunity  to  work  on  it  for  several 
days  at  a time  — not  just  one  period.  She 
understood  better  in  a dass  format  than 
independently.  She  was  very  good  with 
the  use  of  manipulatives.  She  worked 
well  cooperatively.  If  we  had  a checklist 
for  this,  or  at  least  a strategies  list,  then 


her  progress  would  be  much  more 
apparent. 

Though  there  is  no  record,  the  class 
structure  regarding  math  was  re-shaped 
as  a result  of  students  like  A.  Students 
were  taken  off  an  independently-paced 
workbook  diet  and  fed  group  tasks, 
often  with  manipulatives  and  a lot  of 
repetition.  Perhaps  to  the  credit  of  this 
structural  response,  A.  seemed  to  be- 
come much  more  confident  and 
self-aware  in  pursuing  her  math  work. 
She  went  from  thinking  she  could  not 
do  it  to  realizing  she  just  hated  it.  She 
and  other  students  proposed  a structural 
change  because  of  their  frustration  in 
pursuing  math  on  a one-day-a-week 
schedule.  The  result  was  to  focus  on  one 
subject  at  a time,  for  a whole  week  or 
weeks.  This  to  me  is  a clear  advance  in 
learning  self-awareness  and  empower- 
ment— though  it  is  nowhere  recorded. 

These  conclusions  have  made  me 
return  to  the  checklists  and  consider 
how  to  revise  them  to  better  suit  the 
needs  of  the  class  and  the  students.  The 
revision  will  include  more  strategies  and 
learning  behaviors  and  also  be  broken 
into  smaller,  more  measurable  steps. 

WRITING  SAMPLES.  The  writing 
samples  contained  in  A/s  progress 
folder  are  good  indications  of  how  we 
focused  on  process  together,  with  three 
drafts  of  a writing  included  (see  Figures 
3A-D).  Importantly,  this  was  the  one 
thing  in  the  folder  that  A.  could  look  at 
and  clearly  sense  her  progress.  The  first 
sample  we  have,  from  last  fall,  shows  a 
short  corrected  piece  that  still  contains 
some  grammar  and  writing  and  spelling 
mistakes.  The  second  sample  are  all 
three  drafts  of  a piece  she  had  worked 
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BAS  I C MATH  MASTER  CHECK  l_XST 

. dorr 


Cfrnfr£X£NCY Mastery  Date : 

1.  Adding  Whol e Numbers  fc  Money 

^7 : 

2.  Subtracting  Whole  Numbers  & Money 

I 

3.  Multiplying  Whole  Numbers  ft  Money  1 

4.  Dividing  Whole  Numbers  Si  Money  , ^ . .y  \ 

Wi>lC  \ Uai  C • i l 

6,  COMPREHENSIVE  REVIEW  OF  WHOLE  NUMBERS  Sr  MONEY 

7.  Units  of  Measure 

V'"  I5.-1*T'93j 

8.  Meaning  of  Fractions  "7 

( ^Ooc‘.gj„t  otr  1 

1 y : 

Adding /Subtracting  Fractions' 
vy  (Like ^Denominators) 

10.  Addi ng/Subtracting  Fractions 
(Different  Denominators ) 

11.  Multiplying  Fractions 

12.  Dividing  Fractions 

13.  Fraction  Applications 

14.  COMPREHENSIVE  REVIEW  OF  FRACTIONS 

15.  Meaning  of  Decimals 

16.  Adding  & Subtracting  Decimals 

17.  Mul ti plying  Decimals 

18.  Dividing  Decimals 

1 

19.  Decimal  Applications 

20.  COMPREHENSIVE  REVIEW  OF  DECIMALS 

1 

21-  Introduction  to  Equations 

22.  Rati o/Proport ion 

23.  Meaning  of  Percent 

' r 

24.  Percentage  Problems 

1 

25.  Tables  and  Graphs 

^ 1— i-  . 

26.^<peri «ete\  Area.  Volume 

(Rectanales.  Rectanaular  Solids) 

1 

27.  BASIC  HATH  REVIEW 

1 
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on  in  the  Spring.  She  looked  at  this 
during  our  April  conference  and 
pointed  out  to  me  the  progress  she  had 
made:  better  spelling,  a longer  piece, 
clearer  writing.  She  could  also  see  her 
process,  that  her  first  draft  did  not  have 
to  be  perfect  but  that  she  would  have 
several  passes  to  improve  it.  She  was 
very  proud  of  her  progress. 

SELF-ASSESSMENTS.  A/s  December 
self-evaluation  is  the  only  one  I have  on 
file.  Her  comments  are  the  comments  of 

many  different  students — broad  and 
- 1 

ICC  ill  Ig-  l/CWCU. 

"We  learned  alte  about  America  and  In- 
dian... I don't  understand  math  divicion...  I 
wish  to  learned  more  about  riding  and 
writing..." 

These  comments  are  too  broad  to 
indicate  much  progress.  The  strength  of 
this  tool  is  more  in  the  process.  We  do  it 
as  a class.  The  class  discussion  focuses 
on  the  curriculum,  what  has  been  valu- 
able, and  what  we  want  to  focus  on  in 
the  future.  When  we  do  these,  students 
seem  very  involved  in  answering  the 
questions,  too,  which  I think  is  an 
indication  that  taking  the  time  to  reflect 
is  valuable,  even  if  the  record  is  so 
general. 

STUDENT  M. 

M.'s  folder  resembles  a "textbook" 
progress  portfolio.  There  is  a more 
complete  array  of  contracts  and  check- 
lists. Like  A.'s  folder,  there  are  writing 
samples  that  M.  chose  to  include  in  this 
folder,  as  well  as  assessment  tools.  In 
M's  case  these  include  a grammar 
pretest  from  one  of  the  workbooks  we 
use  in  the  class.  Also  there  are  filled  out 


reading  and  writing  behavior  checklists, 
adapted  or  borrowed  from  the  Read/ 
Write  Now  program  in  Springfield,  MA. 

M is  more  familiar  with  her  folder, 
too.  She  has  paged  carefully  through  the 
contents  at  different  occasions. 

LEARNING  CONTRACTS— a useful 
tool.  M's  learning  contract  (see  Figure 
4)  focuses  her  toward  the  External 
Diploma  Program,  a high  school  di- 
ploma program  run  by  the  Boston  Public 
Schools.  This  specific,  external  goal 
provides  helpful  direction  for  both  of  us. 
Hor  Tannarv  list  offers  indeoendent 
activities  for  her  completion.  Goals  also 
include  specific  competencies  to  achieve. 

Reading  Goal  — to  understand  bigger 
words. 

Make  a list  of  words  and  go  over  these 
Read  bigger  words  in  class 
Use  a vocabulary  book  to  build  your 
understanding 

Writing  — 

Continue  the  good  work 
Practice  for  EDP 
Work  on  editing  list 

Grammar  — 

Complex  sentences 
Punctuation — Commas 

Math  — 

Units  of  Measure 
Fractions 

Other  — EDP  or  GED 
take  EDP  pretests  or  plan  to  take  GED  at 
the  end  of  March 

This  contract  has  been  a more  effec- 


The  Tale 

of  the  3 5 
Tools 
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tive  tool  with  M.  than  with  A.,  evident 
in  the  checks  and  comments  that  were 
placed  on  it  during  our  April  confer- 
ence. It  was  used  much  more  as 
intended,  as  a contract,  and  advances 
were  noted  and  commended. 

The  April  contract  is  more  bare.  The 
Reading  section  suggests  again  to 
understand  bigger  words  (her  stated 
goal)  with  weekly  work  in  a vocabulary 
book  as  a suggested  independent  activ- 
ity. The  Grammar  section  is  blank,  and 
the  Writing  section  suggests  only  an  edit 
list.  The  Math  section  lists  specific  math 


• Ll  ***+  f **  r*  Tin  i'Vt  /4  I /I  A_ 

dLUVIUCO  v auuuig  VYXU.I  uiutiwm 


nominators...  whole  numbers/mixed 
numbers/improper  fractions")  as  well 
as  an  asterisk  next  to  the  goal  "Become 
more  Independent." 

What  became  important  in  April, 
however,  was  a checklist  made  for  her 
by  the  EDP  program  assessor.  M had 
taken  the  pretests  and  needed  to  master 
some  math  and  writing  skills  before  she 
could  enroll  in  the  program.  A copy  of 
this  list  was  included  in  the  progress 
portfolio  and  also  in  M's  notebook.  This 
provided  her  with  a lot  of  self-direction 
and  was  referred  to  almost  daily 
throughout  the  cycle. 

This  contract  or  checklist  was  a list  on 
a sheet  of  notebook  paper.  Yet  M held 
on  to  it  for  three  months  while  she 
worked  towards  completing  those  goals. 
As  such,  it  shows  that  forms  don't  have 
to  be  carefully  typeset  or  formatted  to  be 
effective. 

In  our  June  conference,  M.  an- 
nounced that  she  was  almost  complete 
with  her  work  toward  the  EDP.  (The 
checklist  had  provided  her  with  an 
exceptional  tool  for  keeping  herself  on 
target).  Moreover,  her  new  contract 


reads  very  successfully.  Notice  the 
switch  between  her  and  my  voice: 

MATH:  a lot  better;  didn't  know  to 
much  about  fractions  and  decimals  and 
measurements  [before];  [I  now  know  more 
about ] stores  and  how  much  things  are 
going  to  cost — and  also  paying  bills. 

(Working  more  independently<hecking 
yourself  and  re-doing  your  problems  if 
necessary .) 

"I  like  to  do  math  now — seems  to  be  a lot 

one  igr 


(If  you  come  to  a problem  you  don't  under- 
stand, you'll  come  back  to  it  later,  try  to 
figure  out  what  would  make  sense,  go  to  a 
new  page  and  come  back  to  figure  it  out.) 

Notice  that  the  comments  focus  on 
strategies  rather  than  skills  acquisition. 
Also,  the  recording  relies  heavily  on 
self-report.  Also,  it  is  not  a "contract"  in 
the  formal  sense.  It  is  halfway  between 
an  evaluation  and  a goals  list.  It  does  not 
require  a signature,  but  it  is  an  oral 
agreement.  It  is  very  effective  as  a 
record  of  strategies  tried  and  true,  or 
abandoned. 

M/s  5 week  summer  plan  focuses  her 
heavily  on  the  EDP  skills  she  has  yet  to 
master. 

TEACHER'S  LOG— how  difficult  it 
really  was.  I had  an  ongoing  struggle 
with  M.  regarding  independence.  I say 
this  only  to  give  some  background  to 
help  explain  the  sometimes  emotional 
recordings  in  the  teacher's  log  regarding 
M's  progress.  The  reader  should  under- 
stand the  tool's  ability  to  record  how 
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difficult  learning  can  really  be,  a process 
that  does  not  show  up  on  a checklist  or 
in  a grade  or  even  a learning  contract. 

2/1 — "she  doesn't  retain  well.." 

2/3 — "M. — up  to  page  44  (some  diffi- 
culty with  directions)" 

2/8 — "she  said,  T feel  like  I'm  doing 
better  than  before'...  she  has  to  be 
coaxed  through  reducing..." 

2/9 — "worked  on  math — addition  of 
unlike  denominators,  whole  numbers, 
reducing,  and  simplifying— I had  her 
work  with  paper  manipulatives.  She 

liH-lo  f/>  iico  nanor  QVto 

OttlU  Ab  X IV4^/U  vt  UkMV  kV  VUrt,  » 

was  having  trouble  with  simplifying 
forex:  11/8  = 1 3/8.  The  division  step 
keeps  confusing  her.  I tried  to  get  her 
to  do  it  without  division  step— just  do 
it  in  her  head.  But  that  was  still  a 
challenge — she  doesn't  believe  she 
can  do  it." 

2/10 — Grammar  stuff  confused  her — 
directions,  pg.  37.  Has  she  worked  on 
this?  I think  so.  Have  her  check  her 
papers.  She  feels  comfortable  paired 
with  someone,  but  she  wants  [my] 
okay.  Safety  and  security  is  very 
important  to  her. 

3/30— she  successfully  ploughed 
through  2 digit  division — still  wanted 
help,  but  was  able  to  solve  many 
problems  on  her  own 
4/13 — worked  on  fractions — again  id'd 
problems  when  directed,  but  needed 
a lot  of  support  to  get  there.  I gave 
less  than  usual  When  she  worked  on 
another  segment,  she  said  she  didn't 
understand.  I said  read  the  directions. 
She  gave  up  in  a second.  I told  her  to 
take  five  minutes.  She  started  crying 
and  I held  my  ground  and  said  i will 
help  you  in  5 minutes — try  to  figure  it 


out — she  complained  that  the  direc- 
tions were  short.  She  was  right.  I 
showed  her  the  lengthy  example  and 
explanations  that  were  on  the  same 
page.  She  hadn't  realized.  She  apolo- 
gized for  crying.  I told  her  I want  her 
to  work  independently.  She  held  on 
and  got  work  done.  Good  for  her. 

5/10 — focus  on  one  subject  at  a time... 
5/17 — checked  her  own  math  work — 
got  many  of  them  right.. 

Very  emotional  but  clearly  a showing 
of  what  learning  is — an  emotional  and 
whiv  complicated  task  that  does  not 
only  include  skill  acquisition.  This  is 
recorded  in  detail  here.  Note — mostly 
very  spare,  not  a lot  of  details.  The 
detailed  entry  of  2/9  is  helpful,  I am 
sure,  for  the  reader  to  understand  better 
the  difficulty.  But  remember,  this  is  not 
designed  for  an  outside  reader. 

This  process  has  been  discussed  by 
M.  and  me  since.  She  agrees  she  is 
working  more  independently.  Again, 
she  has  not  seen  these  notes.  Would  I 
share  these  with  her  the  way  I would 
share  the  notes  with  A.?  Yes.  I think  it  is 
respectful  of  a student  to  be  up  front 
about  your  teaching  assumptions — if 
done  in  a sensitive  manner. 

Another  obvious  value  of  these  notes 
is  for  my  own  improvement  What 
strategies  have  I used?  What  has 
worked?  What  hasn't?  This  example,  for 
me,  is  one  of  the  most  powerful  of  my 
teaching  beliefs.  Perhaps  there  is  a way 
that  it  can  be  used  too,  in  teacher  shar- 
ing at  my  center,  or  in  a teacher 
self-evaluation. 

CHECKLISTS . M benefited  more  from 
checklists  than  A.  This  was  most  cer- 


Another  obvious 
value  of  these 
notes  is  for  my 
own  improve- 
ment. What 
strategies  have  I 
used?  What  has 
worked?  What 
hasn't? 
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FIGURE  5 
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Comparative  & Superlative 
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SENTENCES 

sentence  fragments \ 


run-o_n_  sentences 

Compound  Verbs 

Double,  Subjects 

Compound  Sentences 

MECHANICS 


Capitalization 

Using  .Commas 

Using. Semicolons 

using  Apostrophes 

Quoted  & Reported. Speech 
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tainly  true  of  the  highly  motivating  EDP 
checklist  that  she  was  using.  A.,  as  an 
upper  level  ESL  student,  was  out  of 
range  of  the  checklists  that  were  in  our 
class  for  higher  level  ABE  math  and 
grammar  skills. 

Over  time,  M.  made  clear  progress  on 
the  math  checklist  (Figure  5),  though 
again,  her  actual  progress  in  self-motiva- 
tion and  learning  strategies  were  still  not 
listed  on  a checklist.  Her  progress  on  a 
grammar  checklist  was  more  ill-defined. 
Her  initial  ability  was  measured  on  a 
workbook  pretest.  The  checklist,  which 
is  patterned  after  the  competencies  that 
were  worked  on  in  the  workbook,  does 
not  record  much  of  M.'s  progress  be- 
cause M/s  work  in  the  book  was 
sporadic.  The  checklist  needed  to  better 
reflect  the  actual  work  she  was  doing  in 
order  to  indicate  progress. 

WRITING  SAMPLES . M/s  writing 
samples  are  much  less  instructive  than 
A/s.  Why?  She  has  three  different 
samples  from  over  the  year  that  she 
included  when  asked  to  choose  some- 
thing. They  do  not  reflect  the  process 
reflected  in  A/s.  They,  to  my  eye,  look 
very  similar  in  terms  of  style  and  ability. 
Why?  When  she  came  into  the  class,  M 
was  already  a competent  writer.  She  has 
the  ability  to  write  long,  personal  narra- 
tive pieces.  Her  spelling  is  very  good. 
Our  focus  only  recently  has  been  on 
critical,  non-narrative  writing.  When 
these  samples  are  included  in  her  folder, 
they  will  indicate  the  culmination  of  a 
lot  of  work.  For  the  meantime,  the  work 
reflects  M/s  competence  but  not  her 
advance. 


SELF-ASSESSMENTS.  M's  December 
self-assessment  (Figure  6)  reports  "we 
learned  about  Columbus  discovering  a 
new  Continent"  and  that  the  most 
meaningful  of  what  we'd  studied  was 
math  and  science.  Her  concerns  about 
the  class  include  that  many  people  come 
to  dass  late  and  this  seems  frustrating  to 
her.  Also,  a cooperative  project  frus- 
trated her  because  "we  could  not  make 
up  our  minds  for  a long  while." 

This  is  vague  in  terms  of  acquired 
skills;  but  valuable  in  terms  of  learning 
attitudes.  It  could  be  concluded  that  M. 
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enjoy  cooperative  projects.  Also,  she 
gave  me  a cue  as  a teacher  that  the  class 
could  be  more  structured — or  at  least 
she  would  prefer  her  learning  experi- 
ence to  be  as  such. 

Her  May  self-assessment  includes  a 
lot  of  self-praise  and  awareness  of 
learning.  "What  I am  most  proud  of  is 
knowing  what  and  who  I am."  "My 
biggest  breakthrough  was  figuring 
things  out  that  I was  not  sure  of  before." 
"My  favorite  class  was  science  because  I 
learned  things  I never  knew  before." 
"The  goals  I did  not  achieve  are  learning 
bigger  words  because  I do  not  under- 
stand them." 

The  difference  in  these  comments  is 
in  part  controlled  by  the  tools:  different 
questions  were  used  on  different  occa- 
sions. The  latter  occasion  seems  to  have 
been  much  more  effective  in  revealing 
learning  attitudes  and  encouraging  self- 
assessment.  Though  her  descriptions  are 
still  a little  vague,  I think  this  is  because 
self-assessment  is  a challenge. 
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I am  trying  to 
understand 
people's  different 
learning  styles: 
whether  they  work 
well  cooperatively 
or  independently ; 
whether  structure 
is  essential  for 
self-motivation. 

I want  to  under- 
stand the 
student's  process. 
I want  to  see — 
and  I want  the 
student  to  see — 
the  complicated 
reality  of  learning. 


REFLECTIONS  ON  MY 
EXPLORATION  INTO  ASSESSMENT 

WHAT  do  I think  I am  assessing? 

Clearly  I am  not  assessing  for  skill 
acquisition  alone.  I am  looking  for 
increases  in  confidence,  independence, 
learning  self-awareness,  ability  to  take 
new  challenges.  I am  trying  to  under- 
stand people's  different  learning  styles: 
whether  they  work  well  cooperatively  or 
independently;  whether  structure  is 
essential  for  self-motivation.  I want  to 
understand  the  student's  process.  I want 
to  see — and  I want  the  student  to  see — 
the  complicated  reality  of  learning. 

Am  I REALLY  assessing  it? 

The  checklists  that  I use  are  skills-based, 
not  strategy-based,  and  thus  do  not 
record  all  the  information  I want. 
Though  I may  speak  to  the  students  of 
their  advances  in  these  affective  areas, 
the  forms  we  use  do  not  necessarily 
speak  the  same  language.  The  teacher's 
log  and  the  learning  contracts  do  record 
the  strategies  and  attitudes,  as  do  the 
self  assessments.  The  writing  samples 
can  show  process  as  well  as  product. 

Activities  need  to  be  ROUTINE  to  show 
progress 

Though  it  is  a valuable  process,  to  sit 
down  with  students  and  discuss  their 
goals,  the  learning  contract  itself  is 
useful  mostly  when  it  is  used  periodi- 
cally, and  updated.  Independent 
activities  are  effective  if  they  are  nar- 
rowly targeted  toward  the  completion 
of  a goal  such  as  entrance  into  the  EDP 
program.  And  I think  independent 
activities  need  to  be  constantly  rein- 
forced— perhaps  a weekly  check-in  time 


with  a student;  or  time  slotted  in  class 
for  independent  work  (neither  of  which 
my  class  had). 

Checklists  need  to  reflect  the  ivork  that  is 
being  conducted  in  the  classroom. 

The  competencies  need  to  be  specific 
and  attainable.  If  possible,  checklists 
could  be  made  for  strategies  and  learn- 
ing attitudes  and  not  just  skills.  They 
need  to  be  referred  to  often  enough  so 
that  they  are  seen  as  a tool  and  not  an 
artifact  from  an  archeological  dig. 

All  this  is  fine.  But  what  is  helpful 
FOR  THE  STUDENT? 

I have  told  you  what  is  valuable  to  me. 
But  only  a few  of  these  tools  and  activi- 
ties seem  to  have  been  genuinely 
appreciated  by  the  students.  The  Writ- 
ing Sample  was  the  only  thing  A.  used 
to  report  her  own  progress.  The  EDP 
checklist  was  the  one  device  that  M. 
seemed  to  "own."  A powerful  trait  of 
alternative  assessment  is  that  it  has  the 
possibility  to  become  the  property  of  the 
students. 

Teaching  logs  are  valuable  for  me, 
the  teacher.  And  valuable  to  show 
process — the  complicatedness  of  the 
learning  negotiations.  If  the  students  get 
to  see  it,  then  it  will  be  helpful  for  them 
too. 

If  the  tools  are  supposed  to  empower 
the  students,  then  the  materials  need  to 
be  in  the  possession  of  the  students. 
Teacher's  logs  need  to  be  provided  and 
HisrusspH  The  checklists  and  contracts 
need  to  be  possessed  by  the  students. 
The  class  needs  to  have  time  and  struc- 
ture to  accommodate  reflecting  and 
utilizing  these  tools. 

It  is  important  for  teachers  to  discuss 
the  use  of  these  tools.  They  are  altema- 
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tive,  which  means  that,  potentially,  each 
class  is  using  different  forms  and  proce- 
dures. In  a sense,  we  are  all  involved  in 
exploring  the  backroads.  Some  of  us 
with  compasses  in  hand,  carefully 
mapping  our  road;  but  probably  more  of 
us  taking  the  sudden  turn  off  and  trying 
to  remember  which  road  led  where. 


The  GOALS  project  has  allowed 
many  of  us  the  resources  to  carefully 
map  our  route,  and  to  compare  our 
travels.  I hope  the  "Atlas"  we  have 
compiled  in  our  Toolkit  will  help  others 
explore  other  roads  and  paths,  and  that 
we  will  all  have  the  opportunity  to 
share,  revise,  and  learn  from  each  other. 
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From  EGAP  to  Beyond 

Adapting  Tools  to  New  Programs 


Prior  to  my  involvement  in  Compo 
nent  3 of  the  GOALS  Project,  the  use 
of  alternative  assessment  in  Community 
Action,  Inc/s  education  programs  was 
confined  primarily  to  one  program. 
Adult  Education  for  the  Homeless.  It 
was  also  slowly  being  developed  for  the 
adult  basic  education  and  English  as  a 
Second  Language  program. 

As  in  many  programs,  the  funding 
source  tended  to  overly  influence  the 
type  of  assessment  done.  There  had 
always  been  a distinction  between  what 
happened  in  the  DOE-funded  programs 
and  the  Job  Training  and  Partnership 
Administration  (JTPAMunded  pro- 
grams. DOE-funded  programs 
encouraged  the  development  of  new 
materials  and  alternative  means  of 
assessing  student  progress.  But  the 
requirements  of  JTPA-funded  programs 
often  limited  how  far  one  could  go 
beyond  traditional  testing  and  the  use  of 
competency-based  materials. 

The  staff  had  begun  experimenting 
how  to  integrate  more  alternative  assess- 
ment techniques  into  the  JTPA 
curriculum  while  still  meeting  contrac- 
tual obligations.  Participation  in 
Component  3 provided  a mechanism  to 
reflect  on  how  to  do  this  better,  what  the 
considerations  were,  and  how  to  con- 
tinue adapting  materials  to  other 
programs. 

As  noted  above.  Community  Action, 
Inc.  has  been  using  an  alternative  assess- 
ment tool  in  its  Adult  Education  for  the 


Homeless  Program  for  three  years.  The 
tool,  the  Educational  Goals  Assessment 
Package  (EGAP)  (see  Germanowski, 
Adventures  in  Assessment,  Volume  I) 
was  originally  developed  for  a DOE- 
funded  adult  program  serving  new 
readers  to  those  ready  for  the  GED. 
Despite  its  original  focus  for  DOE- 
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modifying  EGAP  for  quite  a different 
use,  namely  its  Entry  Employment 
Experience  Program,  a GED  program  for 
youth  who  have  dropped  out  of  school. 
The  EEE  program  is  a JTPA-funded 
youth  program  serving  learners  at  a pre- 
GED  and  GED  level.  These  differences 
influenced  the  modifications  that  were 
made.  I will  briefly  describe  the  EGAP, 
and  then  discuss  how  and  why  it  was 
adapted  for  use  in  the  EEE  program. 


EGAP  ALREADY  MODIFIED 
FOR  ADULT  ED  PROGRAM 

The  EGAP  contains  initial  assessment 
and  goal  setting  tools,  as  well  as  a means 
for  tracking  on-going  learner  progress. 
CAI  made  several  revisions  to  the  EGAP 
for  the  Adult  Education  for  the  Home- 
less Program.  One  revision  was  in 
format.  The  teacher  found  that  learners 
were  not  filling  in  the  goal  for  the  next 
day  on  their  daily  logs.  The  teacher 
increased  the  space  dedicated  to  this  to 
emphasize  its  importance. 

Another  revision  was  the  addition  of 
a checklist  for  reading  to  give  the 
teacher  a better  idea  of  what  people 
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The  teacher 
wanted  learners  to 
be  more  reflective 
of  and  to  take 
more  responsibil- 
ity for  their 
learning.  The 
short-term  nature 
of  the  program 
and  its  required 
outcomes  made  it 
important  for  the 
learners  to  make  a 
strong  commit- 
ment to  following 
through  on  their 
educational  goals. 


have  read  and  to  see  what  they  are 
interested  in  reading  to  facilitate  plan- 
ning instruction  and  to  understand 
motivations. 

The  goals  of  the  EEE  program  are 
to  assist  youths  to  attain  a GED  and  to 
increase  their  reading  by  two  grade 
levels  and  then  one  of  three  other  op- 
tions: to  transfer  to  a skills  training 
program,  enter  higher  education,  or  get 
a job.  The  program  serves  30  youths 
each  year. 

The  funding  source  conducts  an 
initial  educational  assessment  using  the 
ABLE.  Learners  must  score  at  a mini- 
mum 4th  grade  level  in  reading  and 
math  to  be  referred.  If  accepted,  learners 
are  in  the  program  for  an  average  of  six 
months. 

Meeting  contractual  outcomes  is 
imperative  to  re-funding.  Learners 
enrolled  are  reassessed  by  the  EEE 
teacher  in  reading  and  math  using  a 
modified  SRA  math  assessment  and  a 
teacher-developed  reading  test 

NEEDED  BETTER  ASSESSMENT 
TOOL  TO  FOCUS  ON  STRENGTHS, 
NOT  WEAKNESSES 

Missing  from  the  assessment  array 
was  a means  for  the  teacher  to  identify 
learners'  strengths  and  to  more  quickly 
engage  them  in  the  learning  process. 

The  teacher  wanted  learners  to  be  more 
reflective  of  and  to  take  more  responsi- 
bility for  their  learning.  The  short-term 
nature  of  the  program  and  its  required 
outcomes  made  it  important  for  the 
learners  to  make  a strong  commitment 
to  following  through  on  their  educa- 
tional goals.  The  pattern  had  been  that 
learners  dropped  out  or  did  not  com- 
plete if  they  were  not  committed  or  did 


not  assume  responsibility  for  what 
happened  to  themselves  in  the  program. 

The  initial  assessment  performed  by 
the  funding  source  tends  to  focus  more 
on  identifying  learner  weaknesses  and 
how  to  address  those  than  on  identify- 
ing learner  strengths.  The  teacher 
wanted  a more  positive  means  for 
engaging  with  learners  to  demonstrate 
that  she  was  there  not  to  "fix"  the 
learner,  but  to  help  the  learner  build 
on  his/her  strengths. 

The  EEE  teacher  used  the  checklist 
from  the  EGAP  for  this  purpose  for  a 
year,  but  found  this  did  not  work  well 
for  the  learners.  Many  of  the  youths  are 
at  a GED  level  and  felt  some  of  the  lists 
were  silly  and  not  relevant  The  teacher 
also  decided  that  the  EGAP  lists  were 
too  formulaic  for  the  youths.  They  did 
not  see  the  meaning  of  it,  and  saw  the 
EGAP  as  something  the  teacher  was 
making  them  do. 

The  teacher  also  wanted  more  ex- 
tended answers  than  the  "yes,"  "no,"  or 
other  one-word  answers  she  was  receiv- 
ing to  questions.  The  teacher  felt  the 
format  was  too  confusing  to  learners. 
Youths  would  check  topics  just  because 
they  thought  they  needed  to  for  test 
purposes.  When  asked  why  they  had 
checked  a particular  topic,  the  youths 
could  not  explain  their  choices.  The 
teacher  wanted  to  get  the  learners 
thinking  more  and  coming  up  with  new 
ideas  of  what  they  wanted  to  do. 

In  amending  fire  EGAP,  the  teacher 
made  two  major  changes.  First,  the 
teacher  identified  which  topics  the 
learners  had  consistently  checked  off 
from  the  EGAP  lists  and  then  wrote  an 
open-ended  question  which  related  to 
the  topic.  The  teacher  kept  the  EGAP  list 
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for  types  of  reading  done  by  the  student, 
but  added  a section  for  the  learner  to  list 
what  they  had  read  in  each  category. 

Second,  the  teacher  revised  the  daily 
log.  The  EEE  teacher  used  the  EGAFs 
daily  log  for  a year,  but  found  that 
learners  were  not  consistent  in  filling 
out  the  log  daily.  They  either  forgot  or 
did  not  know  what  to  write.  The  teacher 
modified  it  to  a twice-weekly  log. 
Learners  do  a weekly  plan  on  Mondays 
and  — if  they  want  — can  comment  on 
their  progress  throughout  the  week  in  a 
space  provided.  On  Fridays,  each 
learner  writes  about  the  week,  what 
worked  for  her  and  what  she  will  do 
differently  the  following  week.  The 
teacher  will  be  using  this  form  in  the 
fall  Because  the  program  is  time  lim- 
ited, the  teacher  felt  it  important  to  set 


specific  weekly  goals. 

At  the  same  time,  the  teacher  was 
adapting  the  EGAP  for  use  in  the  class- 
room, she  was  also  working  on  revising 
the  curriculum.  The  changes  in  the 
curriculum  included  incorporating  the 
use  of  math  manipulatives  and  hands- 
on  science  lessons.  The  revisions  in  the 
EGAP  to  better  engage  learners  in  their 
own  learning  were  also  being  addressed 
in  the  curriculum. 

Though  the  two  assessment  forms 
are  different  in  format  — and  to  some 
extent  content  — the  ultimate  purpose 
of  both  the  EGAP  and  the  EEE  Initial 
Interview  and  Weekly  Plan  are  the 
same:  to  assist  learners  in  taking  respon- 
sibility for  their  learning  and  to  give 
teachers  information  for  developing 
individualized  instruction. 
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On  Fridays,  each 
learner  writes 
about  the  week, 
what  worked  for 
her  and  what  she 
will  do  differently 
the  following 
week.  Because  the 
program  is  time 
limited,  the 
teacher  felt  it 
important  to  set 
specific  weekly 
goals. 
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ENTRY  EMPLOYMENT  EXPERIENCE  PROGRAM 
EDUCATION  COMPONENTS  INITIAL  INTERVIEW 

Name:, Date: Age: 

first  Language: 

Educational  background: 

Last  school  attended: 

Last  grade  attended: Last  grade  completed: 

Date  last  attended: 

Solf-Rating 

My  reading  skills  are:  Excellent  Good  Fair  Need  Work 
Explain  your  rating: 


My  math  skills  are:  Excellent  Good  Fair  Need  Work 

Explain  your  rating: 


My  writing  skills  are:  Excellent  Good  Fair  Need  Work 

Explain  your  rating: 


The  following  questionnaire  is  designed  to  provide  you  with  an  opportunity  to  communicate  with 
the  education  instructor  about  what  you'd  like  to  learn  and  how  you  leam  best.  The  best  learning 
takes  place  in  a supportive  environment  where  people  are  able  to  discuss  ideas  and  problems  as 
they  arise.  Communication  is  the  mutual  responsibility  of  all  involved  in  the  learning  process  and 
this  questionnaire  will  begin  that  communication  process. 

Career/Work  Goals 

I would  like  to... 

attend  college  to  study 

attend  a training  program  for 

get  a job  as  a 

The  thing  I need  to  work  hardest  on  to  achieve  my  career  and  educational  goals  is 
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Personal  Goals  lf  . , 

The  (bibwing  questions  give  you  an  opportunity  to  describe  other  things  about  yourseJt  besides  your 
educational  and  career  goals.  Please  complete  all  statements  and  give  as  much  information  and 
explanation  as  possible. 

In  my  free  time,  1 like  to 

Some  of  my  characteristics  which  1 like  are 

Some  obstades  that  keep  me  from  doing  things  X d like  to  do  are 

Other  things  1 would  like  to  leam  (please  describe  any  experience  with  the  following) 

managing  money  effectively 

understanding  pricing,  discounts,  etc. 

reqisterinq  to  vote 

aettina  a library  card 

learning  to  use  the  library 

getting  involved  in  community  activities 
learning  about  careers  such  as 
orotectina  mv  health  effectively 

manaaina  time  effectively 

Please  add  at  least  one  other  thing  you  would  like  to  leam 

Please  list  at  least  one  thing  which  you  can  teach  others 

Do  you  leam  better  working  with  others  or  alone?  Please  explain. 

Do  you  leam  best  by  watching  someone  else,  by  reading  about  something,  by  listening  to  someone 
or  by  doing  something  yourself?  Explain,  and  if  possible,  give  an  example  of  something  you  learned 
well  and  haw  you  learned  it. 
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The  Tale 
54  °f  the 
Tools 


Tberecire*no  right  or  wrong  answers  for  the  following  questions.  The  purpose  is  to  identify  possible 
interests  and  learning  styles  for  each  person. 


My  favorite  subject  in  school  was . 


because 


The  most  memorable  thing  I ever  learned  in  school  was 


I've  always  wanted  to  learn  or  do 


Name  at  least  two  things  you've  seen  or  heard  about  and  would  like  to  understand  better 


Would  you  rather  read  a good  story  ora  news  magazine?  Why? 


Would  you  rather  listen  to  an  older  relative  talk  about  your  family  history  or  listen  to  the  news? 
Why? 


Would  you  rather  discuss  personal  and  social  values  and  how  they  fit  into  your  life  or  how  the  body 
works?  Why? 


Name  at  least  two  other  things  you  enjoy  reading  about  or  discussing  (some  examples  might  be 
hobbies,  places  you've  lived,  sports,  current  events  or  news  you're  concerned  about,  health,  movies, 
music,  etc.) 

Describe  a situation  where  you  were  able  to  help  yourself  or  someone  else  accomplish  something 
(some  examples  may  induce  resolving  a conflict,  doing  a budget,  doing  something  positive  tor  your 

health,  etc) 

Have  you  ever  been  unable  to  do  something  vou  really  wanted  to  do  because  you  didn't  have  the 
necessary  skills  or  information?  Please  describe 

Are  there  forms,  labels,  documents,  reference  boob,  maps,  etc.  which  you  would  like  to  learn  more 
about? 


O i VENTURES  IN  ASSESSMENT 


376 


Volame  5:  Fall  1993 


/Tfil  /ifil  /Ifi!  /ifil  /ifil  /ifi!  /ifil  /ifi!  /ifi!  /ifi! 

The  Tale 

/ / ii  /mil  /mu  /mil  /mil  /mu  /nm  /mu  /mil  /mu 

of  the  55 

Toob 

Personal  Readinc 

1 and  Writing 

1 would  like  to  read  re 

>r  enjoyment  (please  list  anything  you've  read 

recently  in  any  category) 

adventure 

mystery 

romance 

history 

sports 

music 

Doetrv 

religion 

horror 

TV  & movies 

true-life  stories 

self-help 

how-to 

hobbies 

parenting 

science  fiction magazines newspapers 


I have  written  or  would  like  to  write: 


poems 

songs 

stories 

ioumal/diarv 

essays 

school  papers 

news  articles 

letters 

advice  to  others 

Other  reading  or  writing  skills  you  have  or  would  like  to  have: 


The  best  thing  I ever  read  was 


The  best  movie  or  TV  program  I ever  saw  was  

I also  enjoy: 

cartooning painting drawing 

singing other t 
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EDUCATIONAL  COALS  PLAN 

Name: Date: 

What  do  you  want  to  accomplish  in  our  doss? 

These  are  my  educational  goals.  I'll  work  on  them  every  day  in  doss: 

1. 

2. 

3.  

4.  

5.  

6.  


WEEKLY  PLAN 

Name: Date:  

My  goals  (or  this  week  are  (indude  page  numbers,  specific  tests,  writing,  as  well  as  understanding 
concepts  and  ideas,  etc.) 

1. 

2. 

3.  

4.  

5.  

Please  (eel  free  to  make  changes  in  your  goals  during  the  week  and  voice  any  concerns,  ques- 
tions, etc.  in  the  spae  below. 

Day  Your  Comments 


Friday:  How  did  your  week  go?  What  strategies  were  helpful  and  what  will  you  do  differently  next 
week? 
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The  Learner's  Log 

Evolution  of  An  Assessment  Tool 


T ntemational  Language  Institute  (ILI) 
JLhas  always  been  extremely  inter- 
ested in  alternative  assessment.  Since 
the  program  began  in  1984,  students 
have  been  asked  to  evaluate  their 
progress  and  their  classes  through 
ongoing  oral  feedback  in  the  classroom, 
individual  conferences  with  students, 
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tions.  Through  in-house  workshops  and 
attendance  at  state-wide  and  national 
conferences,  ILI  staff  have  focused  on 
assessment  and  over  the  years  have 
added  daily  dialogue  journals,  student 
writings,  taped  recordings  of  the  stu- 
dents, and  — the  subject  of  this  article  — 
learner  logs  as  a way  of  looking  at 
student's  progress  through  portfolio 
assessment 

Being  involved  in  Component  #3  has 
been  rewarding  because  it  has  allowed 
ILI  to  study  alternative  assessment  in- 
depth  through  intense  discussions  about 
assessment  with  other  Component  #3 
participants  and  the  chance  to  examine 
tools  that  other  programs  are  using  in 
the  field. 

Meeting  with  participants  of  Compo- 
nent #3  over  the  last  year  and  a half  has 
made  it  quite  apparent  that  alternative 
assessment  is  an  ongoing  process;  tools 
that  ILI  used  last  year  are  not  used  now, 
the  weekly  evaluation  tool  has  changed 
many  times,  new  tools  have  been 
adopted  as  a result  of  sharing  tools  from 
other  programs,  and  we  still  feel  that 
more  tools  need  to  be  developed  for  our 


low-level  ESL  and  ESL  Literacy  classes. 

Hi's  involvement  with  Component 
#3  helped  us  to  refine  our  Learner's  Log, 
and  the  purpose  of  this  paper  is  to 
discuss  the  evolution  of  the  Learner's 
Log  as  an  assessment  tooL 

THE  LEARNER'S  LOG 

At  TT.Tj.  portfolio  assessment  is  ns6cl  to 
measure  progress  by  both  the  student 
and  the  teacher.  Each  student  has  a 
folder  that  is  kept  in  their  classroom.  In 
the  folder  are  the  student's  learner  log  — 
a series  of  assessment  tools  that  are 
stapled  together  (filled  out  on  a weekly 
basis)  and  writings  that  the  students 
have  placed  in  their  folder.  The  student 
tapes  are  kept  in  the  classroom,  and  the 
students  keep  their  dialogue  journals 
during  the  course.  At  the  end  of  the 
course,  everything  in  the  portfolio  is 
kept  in  the  student's  file. 

The  original  components  in  the  first 
Learner's  Log  (Winter  1992)  contained 
an  assessment  tool  which  students  filled 
out  on  a weekly  basis.  Students  re- 
sponded to  the  following  statements: 

♦ Some  things  I have  learned 

4 Some  things  I didn't  understand 

♦ Some  things  I liked 

4 Some  things  I didn't  like 
4 Some  things  I want  to  do  next  week 
(adapted  from  the  Read/Write/Now  ongoing 
assessment  tool,  called  “A  Learning  Log  ," 
Janet  Kelly,  Adventures  in  Assessment, 
Volume  2.) 

The  students  seemed  to  be  intrigued 
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It  quickly  became 
apparent  that  you 
can't  hand  out 
this  neat  little 
booklet  and  expect 
your  students  to 
respond  to  the 
statements  in  the 
detail  desired  by 
the  teacher.  Most 
importantly, 
students  need  to 
know  why  the 
instructor  is 
taking  class  time 
to  discuss 
progress  and 
self-assessment. 


by  the  collection  and  were  receptive  to 
writing  their  thoughts  about  the  course 
at  the  end  of  each  week.  Also  included 
in  the  Learner's  Log  was  a learning 
styles  instrument  from  the  Center  for 
Innovative  Teaching  Experiences.  This 
tool  asked  students  to  respond  to  state- 
ments about  their  learning  styles.  GLI 
only  used  this  tool  in  the  Spring  1992 
program,  as  it  met  mixed  reviews  by  the 
students.  Instructors  are  looking  into 
other  tools  that  help  students  determine 
their  individual  learning  styles.) 

It  quickly  became  apparent  that  you 
can't  hand  out  this  neat  little  booklet  and 
expect  your  students  to  respond  to  the 
statements  in  the  detail  desired  by  the 
teacher.  Most  importantly,  students  need 
to  know  why  the  instructor  is  taking 
class  time  to  discuss  progress  and  self- 
assessment.  Students  need  to 
understand  the  importance  of  looking 
at  their  progress  weekly. 

As  we  read  the  responses  in  the 
Learners'  Logs  HJ  came  back  to  the 
same  questions:  How  can  we  get  stu- 
dents to  look  at  their  learning?  How  can 
we  help  them  see  their  learning  as  a 
process  and  evaluation  of  their  progress 
as  part  of  the  process?  How  can  we  get 
the  students  to  measure  their  own 
progress  rather  than  relying  solely  on 
the  instructor?  How  can  we  get  our 
students  to  answer  the  weekly  evalua- 
tion questions  with  more  than  one 
sentence?  How  can  we  get  our  students 
to  critique  the  classes  and  know  it  isn't 
going  to  be  taken  as  an  insult  to  the 
instructor? 

At  the  end  of  the  Winter  1992  pro- 
gram, ILI  asked  the  students  if  they  felt 
that  the  Learner's  Log  was  useful.  Most 
students  felt  it  was  valuable,  and  we 


decided  to  continue  with  it  and  add 
some  additional  pieces.  The  Spring  1992 
I .earner's  Log  was  presented  to  the 
students  on  the  first  day  of  class  rather 
than  at  the  end  of  the  first  week.  Some 
additional  pieces  required  students  to  fill 
out  and  understand  the  pieces  on  the 
first  day  of  class.  Included  in  the  Spring 
Program's  Learner's  Log  were  the 
following: 

± a page  that  listed  general  course 
information 

± a program  entry  writing  sample 
± a page  divided  into  four  sections  of 
expectations,  goals,  fears,  and  self- 
help  for  the  program  that  students 
responded  to 

* reading  and  viewing  logs. 

In  the  latter,  students  keep  a running 
list  of  what  they  have  read  and  what 
they  have  seen  on  TV,  with  comments. 

The  Learner's  Log  also  took  on  more 
of  a professional  appearance  with  a 
colored  cover  and  end  page. 

At  the  end  of  the  Spring  Program 
students  were  asked  for  feedback  on  the 
Learner's  Log.  Students  still  felt  it  was 
valuable  for  them  to  fill  out,  but  they 
didn't  like  filling  out  the  same  assess- 
ment tool  week  after  week.  The 
instructors  also  found  lower  level  learn- 
ers had  difficulty  responding  to  the 
statements;  very  often  the  statement  of 
"Some  things  I like"  was  answered,  "l 
like  everything."  The  instructors  decided 
that  the  weekly  assessment  was  work- 
ing, but  the  tools  needed  to  be  changed. 

In  the  Summer  1992  program,  the 
original  weekly  assessment  tool  was 
exchanged  for  two  assessment  tools  that 
alternated  weekly.  One  assessment  tool 
asked  each  student  to  list  the  week' s 
activities  and  to  rate  them  for  both  their 
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enjoyment  and  value.  The  teacher  writes 
these  activities  on  the  board;  the  stu- 
dents record  the  specific  activities  in  the 
Learner's  Log  and  rate  them.  The  tool, 
also  asks  the  students  to  answer  the 
following  questions: 

4 Which  activities  were  the  most  help- 
ful? Why? 

4 Which  activities  were  the  least  help- 
ful? Why? 

The  other  assessment  tool  that  was 
used  was  developed  at  the  Community 
Learning  Center.  * This  tool  divides  the 
assessment  into  two  sections:  in  class 
and  out  of  class: 

In  Class: 

1 . No  w I understand 

2.  In  class,  I need  help  with 

Outside  of  Class: 

3.  This  week  I spoke  English 

4. 1 listened  to 

5. 1 read  English 

6. 1 wrote  English. 

The  Summer  1992  students  all 
seemed  to  write  more  on  the  weekly 
evaluation  that  asked  them  to  list  the 
activities  and  rate  them.  It  appeared  that 
the  process  of  listing  all  the  activities 
first  and  then  evaluating  them  was  much 
more  effective  than  having  the  students 
respond  to  statements  about  what  they 
had  leamed/done  in  class.  The  students 
still  expressed  the  desire  to  have  differ- 
ent weekly  assessment  tools.  In  the  Fall 
1992  program  the  Learner's  Log  looked 
similar  to  the  Summer  1992  Learner's 


Log,  but  in  the  12  week  course  the 
weekly  evaluations  alternated  between 
three  assessment  tools  instead  of  two. 
The  instructors  also  came  up  with  an 
additional  final  evaluation  that  con- 
tained the  following  questions: 

4 Has  your  English  improved  since  you 
started  this  course? 

4 Do  you  use  more  English  now  outside 
of  the  classroom?  Where?  With 


whom? 

What  were  your  goals  when  you 
began  the  class,  and  are  you  closer  to 
achieving  them? 
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expected?  How? 

4 Was  the  style  of  teaching  comfortable 
for  you?  Effective  for  you? 

4 What  activities  did  you  like  to  do 
outside  of  class? 

4 What  did  you  learn  about  yourself? 

4 Did  the  class  make  you  fed  good 
about  yourself? 

4 If  you  were  the  instructor,  what 
would  you  do  differently? 

4 Would  you  like  more  interaction  with 
other  ILI  students? 

4 Additional  comments 

The  instructors  were  very  pleased 
with  this  final  evaluation  as  it  sparked  a 
lot  of  discussion  first,  and  students 
wrote  quite  a bit  in  their  answers.  The 
students  were  still  responding  more  to 
the  assessment  tool  that  had  students  list 
the  activities.  It  was  decided  the  Winter 
1993  Learner's  Log  would  only  use  that 
assessment  tool  for  the  weekly  assess- 


ment 


1 See  Appendix  7 in  "Three  by  Three  by  Four  Ongoing  Assessment  at  the  Community 
Learning  Center"  by  Karen  Ebbit,  Priscilla  Lee,  Pam  Nelson,  and  Joann  Wheeler  in 
Adventures  in  Assessment  Volume  2:  Ongoing. 


The  process  of 
listing  all  the 
activities  first  and 
then  evaluating 
them  was  much 
more  effective 
than  having  the 
students  respond 
to  statements 
about  what  they 
had  leamed/done 
in  class. 
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It  is  safe  to  say 
that  the  Learner's 
Log  will  never  be 
a finished  product. 
Our  ongoing 
evaluation  of  our 
Learner's  Log  is 
not  just  within  the 
parameters  of  a 
given  program. 
Plans  are  already 
being  made  to 
create  weekly 
assessment  tools 
for  the  beginning 
ESL  learner  to 
develop  a tool  for 
students  of  all 
levels  that  would 
help  them  look  at 
how  they  learn. 


THE  SECOND  YEAR 

Throughout  the  program,  students 
meet  individually  with  the  instructor  to 
discuss  progress.  In  the  Winter  1993 
Program,  ILI  began  documenting  the 
individual  exit  interview  with  the  in- 
structor. The  instructor  wrote  down 
what  the  student  and  teacher  discussed 
during  the  interview.  The  exit  interview 
is  kept  in  the  Learner's  Log.  The  student 
• interview  record  included  the  following 
areas: 

£ Goals  met 
£ Goals  not  met 
£ Recommendations 
£ Plans  for  future  action 

The  Spring  1993  Learner's  Log 
brought  back  the  alternating  assessment 
tools,  but  this  time  allowed  students  to 
choose  their  weekly  assessment  tooL 
Students  still  preferred  the  assessment 
tool  that  listed  the  activities  first,  but  the 
instructors  were  still  not  satisfied  with 
the  tool  At  the  end  of  the  Spring  1993 
Program  Tim  Rees  and  Pat  Sandoval 
developed  a two-page  weekly  assess- 


ment tool  that  is  being  used  in  the 
Summer  1993  Program  (See  Figure  1). 
The  instructors  have  been  using  the  tool 
for  three  weeks  and  are  very  pleased 
with  the  results. 

The  instructors  also  developed  a new 
' final  student  evaluation  (Figure  2)  which 
will  be  evaluated  by  both  students  and 
instructors  at  the  end  of  the  Summer 
1993  Program. 

During  the  Learner's  Log  evolution, 
TT.T  has  realized  the  importance  of  con- 
tinually reevaluating  the  tools  and  how 
teachers  and  students  alike  must  under- 
stand that  assessment  is  a necessary 
process.  It  is  safe  to  say  that  the 
Learner's  Log  will  never  be  a finished 
product.  Our  ongoing  evaluation  of  our 
Learner's  Log  is  not  just  within  the 
parameters  of  a given  program.  Plans 
are  already  being  made  to  create  weekly 
assessment  tools  for  the  beginning  ESL 
learner  to  develop  a tool  for  students  of 
all  levels  that  would  help  them  look  at 
how  they  learn.  Stay  timed  for  the  next 
installment! 
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FIGURE  I 

WEEKLY  EVALUATION  OF  CLASS  ACTIVITIES 

Date: 

Please  rate  the  week's  activities  for  both  their  enjoyment  and  value.  “I”  is  extremely  low  and  “5"  Is 
extremely  high.  For  example,  you  might  feel  that  a particular  activity  was  a lot  of  fun,  but  you  don't 
think  that  it  helped  you  learn  anything  new.  In  this  case  you  may  rate  it  u4n  or  “5”  in  enjoyment  and 
“ I " or  M2n  in  value. 

Enjoyment  Value  Name  of  activity 


Which  activities  were  the  most  helpful?  Why? 


Which  activities  were  the  least  helpful?  Why? 


Hours  in  class:  

Homework  assignments  given: 

Homework  assignments  completed: 
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Grde  an  answer 


1 spoke  English  with  people  who  speak  my  language. 

YES 

NO 

SOME 

1 wrote  in  my  journal  every  day. 

YES 

NO 

SOME 

1 used  what  1 learned  in  class  when 
1 left  the  school 

YES 

NO 

SOME 

Did  you  speak  or  listen  in  class? 

SPEAK 

BOTH 

LISTEN 

What  did  you  read  in  English  outside  of  class? 

newspapers  magazines  stories  books  other 

What  did  you  watch  in  English  outside  of  class? 

movies  soaps  comedies  news  documentaries  other 

The  BIG  QUESTION!!! 

Write  about  you  and  your  experiences  both  in  and  out  of  class  this  week or  ...how's  it  going? 
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FINAL  STUDENT  EVALUATION 


Teacher’s  name 

Dear  Student:  We  are  very  happy  that  you  have  been  a participant  In  this  program.  We  hope  that  you 
have  enjoyed  your  time  with  us,  and  that  you  have  learned  a lot  of  English.  To  help  us  evaluate  this 
program  and  plan  for  future  programs,  we  ask  you  to  complete  this  form.  We  appreciate  your 
opinions  and  feedback,  so  please  try  to  be  specific.  Thank  you! 

I.  Expectations: 

When  you  came  to  LLI.,  what  did  you  expect  to  learn? 


Did  you  do  what  you  wanted?  Please  comment. 


2.  Motivation: 


Did  you  speak  In  class  when  you  needed  to? 

YES 

NO 

SOME 

Did  you  do  the  assigned  homework? 

YES 

NO 

SOME 

Did  you  try  to  use  what  you  teamed  in  class  when 
you  were  not  in  school? 

YES 

NO 

SOME 

Were  you  on  time  for  class? 

YES 

NO 

SOME 

How  often  were  you  absent  from  class? 

OFTEN 

SOMETIMES 

What  else  did  you  do  to  help  yourself? 


NEVER 


3.Your  Instructor: 

Did  you  understand  your  teacher  when  he/she  spoke? 

Did  you  understand  why  your  teacher  chose  classroom  activities? 

Could  you  talk  with  your  teacher  about  problems  or  concerns  you  had? 


YES 

; NO 

SOME 

YES 

NO 

SOME 

YES 

NO 

SOME 
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4.  Class  atmosphere: 

Did  you  enjoy  spending  time  with  other  students? 

How  much  time  did  you  spend  speaking  your  first  language? 
Please  comment  on  the  class  atmosphere  in  general? 


YES  NO  SOME 
NONE  SOME  LOTS 


5.  Class: 

A How  much  improvement  have  you  made? 


B What  were  the  most  useful  activities  that  helped  you? 

A Grammar 
B 

LITTLE 

SOME 

LOTS 

A Vocabulary: 
B 

LITTLE 

SOME 

LOTS 

A Pronunciation 
B 

LITTLE 

SOME 

LOTS 

A Speaking  ability: 
B 

LITTLE 

SOME 

LOTS 

A Listening: 
B 

LITTLE 

SOME 

LOTS 

A Reading: 
B 

LITTLE 

SOME 

LOTS 

A Writing: 
B 

LITTLE 

SOME 

LOTS 

6.  Culture: 

Did  you  learn  more  about  American  ciiture?  What  helped  you? 


Did  you  learn  more  about  the  cultures  of  students  in  the  class?  YES  NO  A LITTLE 

What  helped  you? 

7.  Last  comments: 

What  do  you  think  about  the  program  In  general? 
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Reflecting  on  the  links  Between  literacy 
Practices  and  Community  Development 


Learning  to  work  collaboratively  with 
shared  visions  and  goals  is  at  the 
heart  of  people  building  better  futures 
for  themselves  and  their  communities. 
Adult  learning  centers  can  foster  the 
skills,  awareness,  knowledge,  and 
experiences  of  working  together  effec- 
tively. Through  group  classes,  manage- 
ment committees,  potlucks,  and  commu- 
nity activities,  learners  may  have  the 
opportunity  to  problem  solve,  offer  their 
opinions,  make  decisions,  advocate,  and 
listen  to  others  of  different  backgrounds. 
As  individuals  experience  being  listeners 
and  being  heard,  they  become  more 
willing  and  able  to  advocate  on  their 
own  behalf  outside  the  four  walls  of  the 
classroom,  be  it  with  spouses,  landlords, 
employers,  teachers,  neighbors,  or 
politicians. 

Many  assessment  tools  used  in 
literacy  programs,  even  those  which 
educators  call  alternative,  usually  focus 
on  how  to  assess  where  students  are  in 
terms  of  their  goals.  Checklists  and 
portfolios  aim  to  give  feedback  to  stu- 
dents and  teachers  about  how  students' 
reading,  writing,  and  numeracy  skills 
have  improved.  While  we  have  found 
these  helpful,  both  of  us  have  recognized 
that  these  accomplishments  are  not  what 
make  us  feel  passionately  about  our 
work. 

We  get  our  energy  for  continuing  as 
practitioners  in  the  field  when  we  see 
Mary  take  the  initiative  to  make  coffee 
for  everyone  when  just  months  before 


she  was  so  shut  down  that  she  did  not 
speak  unless  a teacher  directly  asked  her 
a question.  She  began  huddled  within 
herself;  now  she  gives  us  a hard  time  for 
not  writing  in  our  own  journals.  Or  we 
look  at  Fred  who  started  our  program 
saying  nervously  that  he  hoped  he  could 
get  a private  tutor;  now  he  participates 
on  the  long  range  planning  committee. 
And  Diane  responds  that  the  most 
valuable  thing  she  learned  from  the 
other  students  in  our  creative  writing 
class  is  that  "it's  our  differences  that 
make  it  exciting  and  our  similarities  that 
make  it  safe." 

Or  we  might  take  some  excerpts  from 
Annette's  essay.  It  has  virtually  nothing 
to  do  with  how  much  better  her  writing 
is.  Rather  it  has  everything  to  do  with 
what  she  has  learned  about  herself  and 
others: 

"There  are  two  kinds  of  people.  The  ones 
that  go  out  and  read  and  write.  Have  the 
ability  to  handle  anything  that  comes  their 
way.  Not  afraid  to  do  it.  They  have  the 
confidence  they  need. 

“There  are  also  people  that  stay  back  in  a 
closet.  That  can't  come  out  or  are  afraid  to. 
But  sometimes  we  see  a little  light  in  the 
dark.  We  are  looking  for  more  light.  When 
we  do  we  find  it  very  interesting.  We  found 
out  that  we  too  have  a very  good  mind  and 
feelings  about  things. 

"I  found  out  there  are  many  intelligent 
people  in  the  dark  closet  after  all.  Have  the 
same  confidence.  We  need  to  come  out  of  that 
dark  cbset.  The  light  we  see  feels  so  good." 


by 

Judy  Hofer 
and  Pat  Larson 


The  Literacy  Project: 
Ware  Adult  Education 
Center 

North  Quabbin  Adult 
Education  Center 


Volume  5:  Fall  1993 


ADVENTURES  IN  ASSESSMENT 


The  Tale 
56  of  the 
Tools 


Many 

participants  in 
adult  education 
come  to  us  feeling 
they  are  failures, 
and  that  we  the 
experts  will  fill  up 
their  empty  minds 
with  what  they 
need.  We  want  to 
set  forth  right 
from  the  start  that 
people  coming  to 
an  adult  education 
program  must 
take  an  active  role 
in  their  own 
learning,  and  that 
working  within  a 
group  will 
enhance  that 
process. 


We  need  more  of  it.  It's  like  being  blind 
and  you  can  see. 

We  lack  indicators  to  measure  this 
type  of  growth.  In  fact,  these  measures  of 
an  awareness  that  working  class  people 
also  create  knowledge,  of  increased 
confidence,  feeling  greater  control  over 
one's  life,  participating  in  groups  and 
taking  on  leadership  roles  are  actually 
trivialized  because  we  measure  our 
students'  progress  in  terms  of  narrowly- 
defined  goals.  Many  participants  in 
adult  education  come  to  us  feeling  they 
are  failures,  and  that  we  the  experts  will 
fill  up  their  empty  minds  with  what  they 
need.  We  want  to  set  forth  right  from  the 
start  that  people  coming  to  an  adult 
education  program  must  take  an  active 
role  in  their  own  learning,  and  that 
working  within  a group  will  enhance 
that  process. 

COMING  TOGETHER  AS  A GROUP 

Very  few  of  our  participants  have 
experienced  being  part  of  a group  that 
shared  a sense  of  purpose.  In  rural, 
dying  mill  towns  of  Massachusetts,  with 
virtually  no  public  transportation,  few 
jobs,  no  community  centers,  and  limited 
access  to  social  services,  getting  a GED 
and  improving  reading  and  writing 
skills  barely  scratch  the  surface  of  what 
is  needed  for  people  to  get  out  of  pov- 
erty and  exercise  more  control  over  their 
lives. 

What  is  desperately  needed  is  to 
provide  participants  with  the  opportuni- 
ties, knowledge  and  skills  to  work 
together  cooperatively,  imaginatively, 
with  critical  analyses  and  tools  for 
problem-solving.  Recognition  of  the 
value  of  this  type  of  group  building  is 
growing  in  the  business  world,  with 


quality  circles,  leadership  development, 
and  the  valuing  of  being  a "team 
player."  Yet  where  does  it  enter  adult 
learning  centers?  And  even  if  these  goals 
are  explicit  in  our  mission,  how  do  we 
articulate  these  to  participants,  influence 
their  own  goals  for  themselves,  assess 
progress  in  these  areas,  and  move 
forward  to  then  bring  the  larger  commu- 
nity into  the  classroom  and  the  class- 
room out  to  the  larger  community? 

As  we  work  with  Component  3,  we 
are  attempting  to  develop  language  to 
"look  bade"  and  not  only  assess  indi- 
vidual progress  but  also  to  assess  what 
people  do  as  a group/ community  — 
part  of  the  continuum  of  literacy  devel- 
opment for  a community.  This  is  one 
reason  why  it  seems  important  to  dis- 
cuss how  to  assess  not  only  what  people 
do  as  individuals  but  also  with  groups. 

CREATING  TOOLS  TO  MEASURE 
COMMUNITY,  TOO 

Although  literacy  practices  and 
community  development  may  be  diffi- 
cult to  measure  in  the  traditional  sense 
of  assessment  and  testing,  there  may  be 
indicators  which  show  how  an  adult 
education  center  views  community 
development  and  collaborative  group 
efforts.  The  way  we  began  this  reflection 
was  to  frame  the  discussion  in  terms  of 
such  general  questions  as: 

1.  How  can  we  develop  alternative 
assessment  tools  which  look  beyond 
the  "defidt  model"  of  what  should  an 
individual  be  doing  differently  to  one 
which  looks  at  a person's  place  in 
their  community,  whether  the  class- 
room, the  adult  education  center, 
their  family,  their  neighborhood,  or 
their  town? 
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2.  How  do  we  assess  improvement  and 
change  as  a continuous  process  not 
only  for  the  individual  but  for  the 
community  of  individuals? 

3.  How  does  a program  establish  a 
strong  link  between  the  community, 
its  program,  and  the  participants  in  a 
program? 

4.  How  does  a program  meet  partici- 
pants' personal  goals  while  consider- 
ing the  needs  of  a community?  How 
can  we  assess  the  value  of  both  of 
these  coming  together  so  that  a 
program  achieves  more  "of  its  goals 
more  often  and  more  efficiently"? 
(Stein) 

5.  How  do  we  assess  if  there  is  move- 
ment or  change  on  a continuum  in 
terms  of  community  development  by 
people  participating  in  programs  and 
by  the  program?  (Stein) 

6.  How  do  we  assess  people's  move- 
ment from  individual  goals  to  group 
goals  and  collective  activity?  What 
has  to  happen  to  allow  for  such 
movement? 

7.  How  do  we  bridge  looking  at  how 
individuals  function  in  a group  and 
develop  along  a continuum  to  the 
idea  of  community  development  and 
collective  action? 

8.  In  order  to  assess  community  devel- 
opment, how  do  we  define  leadership 
development?  (Fingeret) 

9.  How  do  we  assess  collective  activity 
of  a group  with  the  group  rather  than 
just  assessment  with  individuals? 

10.  How  do  we  develop  a language  of 
assessment  which  takes  into  account 
the  development  of  community  and 
collaborative  efforts  at  adult  educa- 
tion centers  which  also  speaks  to  the 
needs  of  funders? 


Answering  all  the  above  questions  is 
beyond  the  scope  of  this  discussion 
paper.  And  some  of  the  possible  answers 
may  be  found  only  through  what  Janet 
Isserlis  describes  as  "the  interactive, 
dynamic,  dialogic  roles  of  both  teachers 
and  learners"  in  the  on-going  assessment 
of  daily  classroom  activities  and  other 
center  and  community  activities"  ( Ad- 
ventures in  Assessment,  Vol.  2). 

In  the  Framework  for  Assessing  Program 
Quality  compiled  by  Sondra  Stein  for  the 
Association  for  Community-Based 
Education,  Stein  points  out  a growing 
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groups  in  terms  of  problem-solving. 
Team  problem-solving  is  growing  in  a 
number  of  jobs  to  replace  the  model  of 
workers  being  individual  cogs  in  a 
wheel.  Thus,  participants  in  adult  educa- 
tion programs  may  need  and  want 
experiences  in  "team  efforts".  Stein  says, 
"Instead  of  looking  solely  at  learners  and 
trying  to  figure  out  what  they  need  to  do 
differently  if  we  are  unhappy  with 
program  results,  we  should  also  look  at 
the  conditions  and  processes  that  lead  to 
those  results  and  try  to  figure  out  what 
the  program  needs  to  do  differently." 

By  experiencing  and  participating  in 
groups,  people  may  come  together 
around  group  projects  such  as  commu- 
nity forums  and  other  activities.  For 
example.  Tawny,  who  came  to  The 
Literacy  Project  at  the  end  of  1990  and 
earned  her  GED  in  a few  months,  re- 
turned to  the  Center  in  October,  1991  to 
begin  volunteering  as  a tutor.  Recently 
Tawny  said  that  when  she  first  came  to 
the  Center  she  only  wanted  to  work 
alone  and  focus  on  getting  her  GED. 
"Now  look  at  me.  I am  helping  organize 
workshops,  coordinating  a newsletter  for 


By  experiencing 
and  participating 
in  groups,  people 
may  come  together 
around  group 
projects  such  as 
community 
forums  and  other 
activities. 
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Other  community 
projects  such  as 
organizing 
SHARE  (a  food 
buying  program), 
publishing  a 
community 
magazine,  and 
starting  a legal 
literacy  program 
all  grew  out  of 
discussions  and 
dialogue  which 
began  in  the 
classroom. 


the  Center,  and  getting  the  local 
selectperson  to  come  to  some  of  our 
NEXT  STEPS  meetings." 

Other  community  projects  such  as 
organizing  SHARE  (a  food  buying 
program),  publishing  a community 
magazine,  and  starting  a legal  literacy 
program  all  grew  out  of  discussions  and 
dialogue  which  began  in  the  classroom. 
The  seed  for  the  Legal  Literacy  and 
Advocacy  Project  which  grew  out  of 
several  different  discussions  during  a 
nine  month  period,  according  to  one  of 
the  advocates:  "Somebody's  welfare  was 
cut  off — an  elderly  student  There  was 
no  explanation.  Welfare  just  said.  That's 
it.'  It  was  illegal,  but  she  wasn't  aware 
that  it  was.  We  decided  to  go  about 
helping  her." 

After  six  months  of  asking  questions 
and  talking  with  Legal  Service  lawyers, 
two  people  started  a legal  literacy  and 
advocacy  project  for  people  in  the 
community.  The  group  of  four  people 
who  developed  this  project  learned 
about  keeping  records  and  about  dealing 
with  various  agencies.  They  also  kept  a 
journal  of  their  activities  through  which 
they  talked  to  each  other  about  their 
experiences.  One  participant  said  she 
learned  how  carrying  out  a community 
project  using  teamwork  is  very  different 
from  the  factory  work  she  has  experi- 
enced. 

The  stops  and  starts  for  such  commu- 
nity projects  illustrate  that  progress  is 
not  always  linear,  and  that  it  takes  time. 
For  many  people  it  may  be  recursive  and 
cyclical  with  movement  in  and  out  of  the 
private  and  public  realms  (Fingeret).  For 
example,  as  we  see  Donna  begin  to 
speak  in  a group  at  the  Center  and  break 
through  the  silence  on  an  individual 


level,  we  also  see  the  discussions  in  her 
class  move  toward  community  issues 
such  as  lack  of  jobs  and  lack  of  public 
transportation.  In  time,  Donna  wrote  a 
letter  to  the  local  newspaper  about  the 
need  for  public  transportation.  At  first  it 
appeared  that  this  was  an  isolated 
writing  activity.  But  months  later  when 
nothing  happened,  Donna  talked  about 
the  issue  in  a group  again  and  began 
writing  letters  to  public  officials.  She 
eventually  organized  a group  meeting 
with  the  person  directing  the  regional 
transit  authority.  Thus,  there  is  move- 
ment and  change  both  on  the  individual 
level  and  the  collective  level  as  a group 
comes  together  to  speak  out  in  the  larger 
community.  All  this  takes  time. 

For  now,  our  challenge  as  educators 
and  facilitators  is  to  negotiate  linking 
literacy  practices  to  community  develop- 
ment so  that  students  — and  staff — can 
benefit  from  a common  sense  of  pur- 
pose. 
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FIGURE  I 

WARE  ADULT  EDUCATION  CENTER 
END  OF  CYCLE  EVALUATION  FOR  WRITING  GROUP 

WHAT:  This  assessment  tool  Is  used  to  assess  the  Individual's  progress  In  group  participation,  learning 
from  others  n groups,  and  the  mechanics  and  process  of  writing  as  well  as  to  provide  the  teacher  with 
feedback  for  the  class. 

HOW:  At  the  end  of  the  three-month  writing  group,  each  participant  filled  out  this  assessment  form  in 
our  last  class.  First  I read  through  the  entire  evaluation  to  clarify  and  answer  questions.  To  respond  to 
question  #1 , as  a group,  we  brainstormed  the  qualities  we  felt  were  most  important  to  us  an  a whole. 
From  that  list,  indivdiuals  chose  their  own  qualities  to  reflect  upon  furher.  After  completing  the  form, 
we  shared  our  responses  with  each  other. 

WHY:  We  used  this  form  to  assess  ourselves,  provide  each  other  with  feedback,  and  make  recommen- 
dations for  future  groups. 

WHEN:  This  is  to  be  used  as  * symmative  assessment  at  the  Last  class. 

POPULATION:  This  tool  is  used  in  a mixed  group  with  very  beginning  writers  and  others  who  are 
going  on  to  community  colleges. 

1 . List  four  qualities  that  you  think  are  particularly  important  to  participating  in  a group  class. 

2.  Do  you  feel  you  have  improved  in  these  areas  since  being  in  this  class? 

3.  What  did  you  learn  from  the  other  students  in  the  class? 

4.  Assess  yourself  in  the  following  areas: 

Mechanics  of  Writing  Not  at  all 

I .  Puncutation 
1 Spelling 

3.  Grammar 

4.  Organization:  main  idea,  paragrphs,  sentences 

Writing  Process 

1 . Ability  to  write  creatively 

2.  Writing  becomes  a part  of  my  life 

3.  Writring  helps  me  understand  myself  better 

4.  Carily  of  writing 

5.  Can  express  what  1 want  to  say  In  writing 

6.  Willingness  to  share  writing 

7.  Enjoyment  of  writing 

8.  Fluidity  of  writing 

5.  What  other  areas  do  you  feel  you've  improved  In?  Expand  on  these  and  the  above  checklist. 


Improvement 

Somewhat  Quite  a bit  A lot 
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6.  Fill  out  the  following  paragraph: 

Next  time  Pm  in  a das * or  group  ike  this,  I hope  /... 

7.  What  was  your  favorite  part(s)  of  this  class?  Or  what  do  you  think  was  best  about  this  class? 

8.  fill  out  this  paragraph,  beginning: 

Judy,  the  next  time  you  offer  this  course,  you  should... 

9.  Please  describe  this  course  as  if  you  were  talking  to  a friend.  Would  you  tell  him/her  to  take  it?  Why 
or  why  not? 

1 0.  Any  other  comments,  suggestions,  thoughts? 

1 1 . Are  you  continuing  into  the  next  cycle?  If  yes,  what  would  you  like  to  see  happen? 
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Analyzing  Self  Evaluation  Checklists: 
A Starting  Point  for  Dialogue  


The  Workplace  Education  Project  of 
the  Labor  Education  Center  offers 
ESL,  Pre-GED  and  GED  classes  in  New 
Bedford  and  Fall  River.  Our  students  are 
primarily  Portuguese,  although  some 
are  Cape  Verdean  or  Polish.  About  half 
our  students  are  employed  as  stitchers 
or  machine  operators.  Some  work  for 
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unemployed  students  is  rising. 

Our  program  worked  on  alternative 
assessment  in  the  GOALS  Accountabil- 
ity Project,  focusing  on  different  options 
to  document  progress  through  portfolio 
assessment.  In  this  process  we  revised 
our  old  assessment  tools,  and  adapted 
and  created  new  ones.  We  field-tested 
these  tools  and  then  discussed  their 
usefulness  for  our  particular  group  of 
learners  and  our  project. 

One  of  the  tools  in  our  portfolio  is  a 
progress  checklist  which  we  adapted 
from  Janet  Kelly's  forms.  (Ed.  note:  see 
new  mother  Kelly's  article  on  "blank  blank," 
on  page  xx.)  To  make  the  checklist  suit- 
able for  second  language  learners,  we 
based  the  items  on  our  students'  goals, 
as  expressed  over  the  last  few  years.  The 
form  asks  learners  to  reflect  on  the 
frequency  of  their  use  of  speaking, 
reading,  and  writing  skills  in  various 
settings.  Students  can  choose  from  "not 
yet",  "a  little",  "sometimes",  and  "usu- 
ally". 

The  progress  checklist  is  two  pages 
long  and  initially  takes  a long  time  to  fill 
out.  As  students  become  more  familiar 


with  it,  however,  the  process  becomes 
quicker.  Our  program  has  multi-level 
classes  and  therefore  the  advanced  r 
students  can  usually  help  the  beginners. 
Students  complete  this  checklist  every 
two  to  three  months.  We  thought  that 
this  interval  would  make  it  more  likely 
that  learners  would  have  made  progress 
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themselves. 

We  expected  this  self-evaluation  tool 
to  show  the  students  their  improve- 
ments in  English.  This  kind  of 
self-assessment  usually  helps  students 
think  about  their  own  progress  and  is 
also  a step  in  taking  responsibility  for 
their  learning.  We  expected  to  see 
general  movement  toward  increasing 
use  of  English  in  most  of  the  skill  areas 
in  most  settings.  This  was  not  always  the 
case,  as  the  following  example  illus- 
trates. 

One  student,  Maria,  completed  the 


form  on  November  9, 1992.  On  all  but 


the  last  skill  Maria  checked  "a  little". 


She  felt  that  she  didn't  understand  and 


couldn't  use  the  conditional  tense  yet 
(Figure  1,  next  page). 

Maria  completed  this  form  again  on 
January  25, 1993.  Usually  students  don't 
see  their  initial  form  before  they  com- 
plete a new  one.  This  way  they  have  to 
think  about  how  they  fed  about  their 
progress  at  that  particular  time  (and  not 
just  replicate  what  they  checked  off  last 
time).  If  we  compare  the  responses  from 
November  to  January,  however,  we  see 
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FIGURE  1 

Name:  Maria  Date:  1 1-9-92 

Usually 

Sometimes  A Little 

Not  Yet 

1 read  my  writing  to  others. 

✓ 

I write  answers  after  reading. 

✓ 

1 understand  when  someone  talks  to  me  in  English. 

✓ 

I can  talk  about  what  is  happening  now  (present). 

✓ 

I can  talk  about  what  happened  before  (past). 

✓ 

I can  talk  about  what  will  happen  (future). 

✓ 

1 can  talk  about  what  might  happen  (conditional). 

✓ 

Name:  Maria  Date:  1-25-93 

Usually 

Sometimes  A Little 

Not  Yet 

I read  my  writing  to  others. 

✓ 

I write  answers  after  reading. 

✓ 

1 understand  when  someone  talks  to  me  in  English. 

✓ 

I can  talk  about  what  is  happening  now  (present). 

✓ 

I can  talk  about  what  happened  before  (past). 

✓ 

1 can  talk  about  what  will  happen  (future). 

✓ 

1 can  talk  about  what  might  happen  (conditional). 

✓ 

FIGURE  2 

Name:  Cariotta  Date:  1 1-9-92 

Usually 

Sometimes 

A Little 

Not  Yet 

1 speak  English  with  my  friends  at  work. 

✓ 

1 speak  English  with  my  supervisor. 

✓ 

1 speak  with  my  union  representative  in  English. 

✓ 

1 speak  English  with  my  family. 

✓ 

1 speak  English  at  the  store. 

✓ 

1 speak  English  at  the  doctor’s  office. 

✓ 

1 speak  English  with  my  neighbors. 

Name:  Cariotta  Date:  (-25-93 

Usually 

Sometimes 

A Little 

Not  Yet 

1 speak  English  with  my  friends  at  work. 

✓ 

1 speak  English  with  my  supervisor. 

✓ 

1 speak  with  my  union  representative  in  English. 

✓ 

1 speak  English  with  my  family. 

✓ 

I speak  English  at  the  store. 

✓ 

I speak  English  at  the  doctor's  office. 

✓ 

I speak  English  with  my  neighbors. 

✓ 
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Maria  felt  that  she  made  progress  in  all 
but  one  category.  Looking  at  grammar, 
she  now  feels  she  uses  the  tenses  "some- 
times" rather  than  "a  little".  For  the 
conditional  tense,  she  feels  she  is  now 
using  it  "a  little"  as  opposed  to  not  at  all. 
She  doesn't  feel  that  she  understands 
English  any  better. 

After  completing  the  form  in  January, 
Maria  looked  at  her  form  from  Novem- 
ber to  realize  she  felt  she  was  making 
progress.  Maria's  progress  checklists 
confirmed  our  expectations.  Half  the 
class  displayed  response  patterns  similar 
to 

The  responses  of  the  other  half  of  the 
class,  however,  didn't  change  with  time 
in  the  anticipated  way  and  puzzled  us. 
Carlota  is  one  example  from  this  group 
(Figure  2).  Her  form  from  November  9, 
1992  shows  that  she  speaks  with  her 
friends  at  work  "a  little"  but  not  with 
her  supervisor  or  her  union  representa- 
tive. She  also  speaks  "a  little"  with  her 
family  in  English  but  only  "sometimes" 
at  the  store  and  never  at  the  doctor's. 

She  didn't  check  off  at  all  how  fre- 
quently she  talks  to  her  neighbors. 

Carlota  filled  out  this  form  again  at 
the  end  of  January.  Most  of  her  ratings 
stayed  the  same.  For  example,  she  still 
doesn't  speak  to  her  supervisor  or  her 
union  representative.  She  moved  from 
"a  little"  to  "not  yet"  concerning  speak- 
ing with  friends  at  work.  However,  this 
time  she  says  she  speaks  with  both  her 
neighbors  and  the  doctor  "sometimes". 
She  also  feels  she  is  speaking  English 
more  with  her  family. 

FORCING  US  TO  RECONSIDER 

Examining  the  inconsistent  results, 
my  colleagues  and  I reconsidered  the 


value  of  this  tool.  Our  dissatisfaction 
stemmed  from  the  differing  response 
patterns  we  saw  in  some  students'  forms 
and  that  we  had  difficulty  in  interpret- 
ing. Our  main  question  was:  Is  it 
important  that  students  move  in  a linear 
progression  in  their  language  acquisi- 
tion or  English  usage?  If  we  are  going  to 
use  a subjective  tool,  shouldn't  we 
accept  all  subjective  responses?  What 
could  we  learn  from  students'  self- 
perceptions of  their  progress?  Our 
conversation  reinforced  and  strength- 
ened our  conviction  that  we  valued 
knowine  how  the  learner  felt  about  his 
or  her  progress.  We  decided,  however, 
that  we  also  needed  to  probe  deeper  into 
the  reasons  for  the  responses  that  indi- 
cated a lack  of  progress. 

Going  back  to  the  classroom,  I talked 
to  Carlota  about  her  form  and  this 
conversation  clarified  some  of  her 
responses.  Her  increase  in  talking  to  the 
doctor  from  "not  yet"  to  "sometimes" 
was  a reflection  of  the  changed  family 
situation.  One  of  her  family  members 
had  some  serious  health  problems  and 
she  had  to  accompany  him  to  the 
doctor's  office  often.  If  I had  known  this, 
I could  have  introduced  more  health- 
related  material  into  the  class  or  could 
have  had  her  work  on  it  individually. 
Carlota  doesn't  speak  to  her  supervisor 
or  union  representative  in  English 
because  they  are  Portuguese.  Her  job  in 
a predominantly  Portuguese-speaking 
workplace  makes  it  harder  for  her  to  use 
English  in  that  setting.  In  retrospect,  she 
and  I could  then  have  reflected  on  how 
she  feels  about  this  situation.  If  she  had 
preferred  to  speak  more  English  at 
work,  we  could  have  thought  about 
some  strategies  to  do  so.  We  could  also 
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Knowing  how 
learners  feel  about 
their  progress 
enables  us  to  find 
a meaningful 
starting  point  for 
dialogue.  If  big 
gaps  exist  between 
their  perceptions 
of  their  abilities 
and  ours,  then  it 
is  important  to 
"dig  deeper"  into 
the  "why." 


have  concentrated  on  how  she  could 
practice  her  English  in  other  settings. 

Knowing  how  learners  feel  about 
their  progress  enables  us  to  find  a 
meaningful  starting  point  for  dialogue. 
If  big  gaps  exist  between  their  percep- 
tions of  their  abilities  and  ours,  then  it 
is  important  to  "dig  deeper"  into  the 
"why."  For  example,  the  most  advanced 
student  in  my  ESL  class  consistently 
rates  herself  very  low.  I hope  it  is  help- 
ful for  her  to  compare  her  subjective 
ratings  with  other  more  objective  mea- 
sures to  see  that  she  is  progressing. 
Knowing  how  she  feels  about  her  lan- 
guage development,  I can  make  sure 
that  I point  out  to  her  specifically  and 
often  the  gains  she  is  making.  More 
importantly,  I need  to  involve  her  more 
consciously  in  identifying  progress  in 
her  reading  and  writing  since  these  are 
her  strongest  skill  areas.  Ultimately, 
only  she  can  convince  herself  of  her 
progress  but  I hope  that  these  dialogues 
about  our  different  perceptions  could 
contribute  to  a better  self-image  of 
herself  as  a learner. 

In  contrast  to  the  above  student,  I 
had  another  learner  in  my  class  who 
rated  himself  consistently  high  with  this 
tool,  even  from  the  beginning  of  the 
class.  He  checked  off  "usually"  for  most 
of  the  settings  and  skill  areas.  Although 


his  spoken  English  was  rather  fluent,  my 
assessment  of  his  reading  and  writing 
skills  did  not  show  the  same  skill  level 
he  felt  he  had.  So  far,  I have  not  figured 
out  how  to  discuss  our  different  percep- 
tions. I know  this  will  be  necessary  for 
him  to  adequately  understand  where 
and  how  he  needs  to  work  on  reading 
and  writing. 

IS  PROGRESS  ALWAYS  LINEAR? 

Initially,  we  assumed  that  progress 
would  be  a linear  movement  from  lower 
to  higher  English  usage  in  all  skill  areas 
and  in  all  settings.  Now  we  see  that 
progress  is  not  linear  but  rather  moves 
in  jumps  and  starts  and  is  based  on  the 
contextual  conditions.  For  example,  you 
can't  expect  an  increase  in  speaking 
English  with  your  supervisor  if  your 
supervisor  speaks  the  same  language 
you  do.  You  can  expect  a change  in 
speaking  English  at  the  doctor's  if  there 
is  an  increased  need  to  do  so. 

We  need  more  experience  vising  this 
checklist,  especially  involving  students, 
to  develop  its  potential.  But  it  is  clear 
that  to  interpret  the  results  of  this  tool, 
additional  dialogue  with  students  is 
required.  In  fact,  the  dialogue  which 
emerges  from  this  process  is  probably 
the  most  important  outcome  of  this  tooL 
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Reflections  on  On-going  Assessment: 
Documenting  Self-Esteem  and  More 


...To  have  good  dream  when  everyone  has  the  same  chance  to  grow,  to  learn,  and  to  love... 

To  have  good  dream  that  instead  of  just  giving  people  material  things,  we  also  teach  them  to 
help  themelves...To  have  these  good  dreams  but  most  of  all  to  see  all  these  good  dream  come 
true  for  all. 

Pat  L.F.,  Sharing  Our  Thoughts 


In  the  summer  of  1991,  Pat  came  to 

class.  She  was  a verv  timid,  anxious 
¥ 

woman  who  told  me  she  had  recently 
been  laid  off  and  wanted  a GED  to  help 
improve  her  chances  of  finding  work. 
She  asked  if  she  could  meet  with  an 
individual  tutor  since  groups  made  her 
nervous.  I explained  that  our  program 
could  not  provide  this  service  but 
invited  her  to  observe  a class.  Although 
she  was  hesitant,  she  did  sit  in  the  back 
of  the  room  and  watch. 

Pat  came  to  the  next  class  and  joined 
us  at  the  table.  She  listened  intently  and 
at  times  seemed  as  though  she  wanted 
to  speak.  After  a few  more  sessions,  Pat 
began  to  participate  in  classes.  Within  a 
few  months,  she  was  an  active  member 
of  the  group  and  was  encouraging 
others  in  their  efforts  to  leam.  Since  that 
evening  two  years  ago,  Pat  has  moved 
from  the  Pre-GED  to  the  GED  class,  has 
had  three  pieces  published  in  Sharing 
Our  Thoughts,  the  SABES  southeast 
learner-generated  magazine,  has 
worked  with  a group  of  five  learners  to 
write  and  produce  a book  about  envi- 
ronmental issues,  and  has  presented  her 
work  to  a group  of  learners  and  teachers 
who  were  interested  in  doing  similar 


projects.  Most  recently,  Pat  attended  a 
Student  Advisory  Board  meeting  to 
make  decisions  about  the  classes  and  the 
project 

While  Pat's  development  of  self- 
esteem and  sense  of  community  are 
apparent  from  her.  accomplishments  and 
conversations  we  have  had,  I am 
troubled  by  the  fact  that  the  assessment 
tools  and  procedures  vised  in  our  pro- 
gram do  not  reflect  these  achievements 
adequately  back  to  the  learner.  This 
concern  was  highlighted  for  me  when 
Pat  recently  experienced  personal 
problems  which  prompted  her  to  con- 
sider leaving  class.  As  she  struggled 
with  family  issues  and  missed  quite  a bit 
of  class,  I tried  to  convince  her  that  she 
had  come  too  far  to  give  up  on  her  goal 
of  passing  the  GED  test  As  I turned  to 
her  reading  and  writing  folders  and 
journals,  I felt  that  I did  not  have 
enough  tools  to  reflect  her  progress  and 
to  encourage  her  to  persist  I became 
disillusioned  with  the  assessment  tools 
and  procedures  we  had  been  using  and 
began  to  question  whether  the  tools 
were  truly  providing  information  which 
is  important  I wondered  how  they 
could  be  revised  to  offer  more  pertinent 
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I suspected  Pat 
would  value  this 
tool  because  she  is 
a prolific  writer 
and  uses  the  folder 
faithfully  to  store 
work  and  to  order 
drafts.  She  con- 
firmed this  by 
noting  that  the 
writing  folder  "is 
a good  idea.  It  lets 
the  teacher  know 
how  much  we  can 
do  and  under- 
stand." 


data  or  even  whether  they  should  be 
discarded. 

Fortunately,  Pat  did  return  to  class 
after  working  out  some  of  her  problems. 
I was  determined  to  discuss  this  matter 
of  assessment  with  Pat  and  the  other 
members  of  class  to  learn  how  they  feel 
about  the  various  tools  we  use  to  assess 
progress.  In  particular,  I wanted  to 
know  if  the  folders  and  journals  demon- 
strate progress  towards  reaching 
individual  goals.  I wanted  to  learn  how 
we  could  use  these  folders  and  journals 
more  effectively  or  if  we  should  use 
them  at  all.  I also  hoped  to  learn  if  Pat 
and  the  others  felt  they  were  making 
progress  and  how  they  knew  when  they 
were  progressing.  Finally,  I hoped  to 
measure  the  development  of  a commu- 
nity in  the  class  and  to  discover  ways  to 
record  increased  self-esteem. 

ASSESSING  OUR 
ASSESSMENT  TOOLS 

In  this  article,  I focus  on  Pat's  experi- 
ence to  represent  my  general 
dissatisfaction  with  the  assessment 
procedures  we  use. 

In  the  pre-GED  and  GED  classes, 
teachers  and  learners  periodically  set 
learning  goals  and  assess  progress. 
Presently,  this  on-going  assessment 
involves  four  components. 

Writing  Folder: 

4 Topics  to  Write  About 
4 Writing  Skills 
Reading  Folder: 

4 Reading  Progress  Checklist 
4 Reading  Log 
Student  Journal 
Teacher  Log 

Writing  Folders.  Each  student  in  the 


class  keeps  his  or  her  writing  in  a writ- 
ing folder.  Topics  to  Write  About  and 
Writing  Skills  forms  are  also  included.  I 
attempt  to  meet  with  each  learner 
monthly  to  discuss  the  contents  of  the 
form.  In  reality,  our  conferences  usually 
occur  every  other  month. 

During  the  conference,  the  student 
and  I review  his  or  her  self-assigned 
writing  goals.  Students'  goals  are  often 
to  master  specific  writing  skills  such  as 
revising  after  writing  a first  draft  or 
editing  run-on  sentences.  Sometimes  the 
goals  are  more  personal  such  as  writing 
a letter  to  a friend.  Together  we  deter- 
mine whether  the  goals  were  met  or  if 
more  work  is  required.  If  the  goals  were 
accomplished,  new  writing  goals  are  set. 
During  this  meeting,  we  also  look  at 
samples  of  the  learner's  writing. 

I suspected  Pat  would  value  this  tool 
because  she  is  a prolific  writer  and  uses 
the  folder  faithfully  to  store  work  and  to 
order  drafts.  She  confirmed  this  by 
noting  that  the  writing  folder  "is  a good 
idea.  It  lets  the  teacher  know  how  much 
we  can  do  and  understand."  This  an- 
swer was  surprising  for  I had  always 
intended  the  folders  to  be  used  by 
learners  to  track  their  own  progress. 
When  I mentioned  this  to  Pat,  she  said 
that  "the  folders  do  let  us  know  how  far 
along  we're  getting.  It  lets  me  know  I'm 
making  progress  because  I am  doing 
more  skills."  She  added  that  looking  at 
all  of  the  drafts  in  the  folder  reminded 
her  that  "I  did  get  to  write  quite  a bit" 
She  then  added  that  she  enjoys  reading 
my  individualized  responses  to  her 
work,  which  include  compliments, 
questions,  and  suggestions  for  revision. 
She  said  that  she  likes  to  know  what  she 
is  doing  well  and  the  areas  that  she 
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personally  needs  to  address. 

From  our  discussion,  I realized  the 
value  of  reviewing  the  contents  of  the 
folder  regularly  and  became  determined 
to  have  writing  conferences  on  a more 
consistent,  monthly  basis.  I found  that  I 
had  a good  sense  of  the  progress  the 
learners  were  making  because  I re- 
viewed and  responded  to  much  of  their 
writing.  It  is  necessary  to  have  more 
conferences  however,  so  the  students 
become  more  comfortable  with  using  the 
writing  folder  as  a tool  to  help  them 
measure  their  own  progress.  During  the 
conferences,  they  will  have  more  time  to 
review  their  old  writing  and  compare  it 
to  newer  work.  This  will  allow  them  to 
better  observe  their  own  progress  and  to 
identify  their  strengths  and  weaknesses. 
It  became  dear  to  me  that  the  folders  can 
serve  a useful  purpose,  but  adequate 
time  for  review  is  essential 

Reading  Folders.  Reading  Progress 
Checklists  and  Reading  Logs  are  induded 
in  the  Reading  Folder.  Again,  learners 
are  supposed  to  meet  on  a monthly  basis 
with  me  to  review  progress  and  to 
discuss  their  reading  experiences. 
Reading  goals  are  reviewed  and  if  the 
learner  has  met  his  or  her  goals,  new 
ones  are  established.  The  checklist  we 
use  was  adapted  from  one  developed  by 
Janet  Kelly  from  Read/Write/Now  (see 
Adventures  in  Assessment,  Volume  1). 
Learners  may  also  record  material  they 
read  in  or  out  of  class  which  is  of  par- 
ticular interest  or  importance  to  them. 

When  meeting  with  Pat,  I asked  why 
she  had  not  used  this  folder  as  regularly 
as  the  writing  folder.  Pat  responded  that 
she  still  couldn't  do  all  the  reading 
strategies  and  that  some  were  confusing. 


Listening  to  Pat,  I was  reminded  of  a 
comment  Susan  Lytle  made  at  a work- 
shop I had  recently  attended.  She  asked, 
"Where  did  these  objectives  come  from 
on  the  checklists?"  I realized  that  I had 
determined  that  these  reading  strategies 
were  important  for  comprehension  and  I 
emphasize  them  in  instruction.  I had  not 
asked  the  learners  to  develop  this  list  of 
strategies  with  me,  however,  and  there- 
fore risked  asking  them  to  measure 
themselves  against  objectives  which 
may  not  be  realistic  goals  for  them  or 
which  simply  may  not  be  their  goals. 

In  the  future,  I will  introduce  a new 
reading  comprehension  strategy  and  ask 
learners  to  experiment  with  it  for  a 
couple  of  weeks.  I will  then  ask  them  to 
reflect  on  whether  the  strategy  is  useful 
for  them  and  to  determine  whether  this 
strategy  should  be  induded  in  their 
personal  checklists.  It  is  also  my  inten- 
tion to  work  with  learners  to  reflect 
more  on  what  helps  them  to  compre- 
hend what  they  read  and  to  develop 
more  individualized  reading  goals. 

When  I asked  Pat  if  this  folder  was 
at  all  useful,  she  said  that  "It' s good  to 
have  these  charts.  It  lets  the  instructor 
see  what's  going  on  and  how  far  they7 re 
going."  Again,  I was  struck  by  the  fact 
that  she  viewed  the  reading  folder  as  a 
tool  to  be  used  by  the  teacher.  She  did 
not  seem  to  realize  that  she  could  gain 
useful  information  from  reviewing  it  as 
well  When  we  reviewed  two  progress 
checklists,  one  from  January  and  one 
from  March,  she  observed  that  "I  can  see 
I'm  making  progress.  These  folders  are 
good  if  students  get  to  check  back  on  old 
papers."  This  comment  highlighted  a 
major  concern  I have  had  when  using 
the  reading  folder.  Because  I believed 
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1 gain  valuable 
information  from 
reading  journal 
entries.  I learn 
when  materials  or 
learning  methods 
were  useful  and 
when  students  feel 
they  need  more 
review.  I also  get 
insight  into  the 
personal  lives  of 
the  students  so 
that  I better 
understand  their 
goals,  interests, 
and  difficulties. 


that  the  students  should  keep  the  folders 
with  them  to  review  their  own  progress, 

I asked  them  to  use  the  folders  at  home 
and  to  bring  them  to  class.  Frequently 
when  it  was  time  for  reading  confer- 
ences, however,  students  would  have 
left  their  folders  at  home.  They  would 
complete  a new  Progress  Checklist  but 
would  not  have  past  data  with  which  to 
compare  their  answers.  I would  remind 
them  to  review  the  folder  at  home  or  to 
bring  it  to  the  next  class,  but  this  did  not 
always  happen.  As  in  the  case  of  Pat,  I 
realized  I had  not  adequately  assisted 
her  in  reviewing  the  data  which  demon- 
strated her  progress  and  became 
determined  to  allot  more  time  for  this 
analysis.  After  discussing  this  issue  with 
my  colleagues  during  a staff  meeting,  I 
have  decided  to  keep  all  folders  in  class 
so  that  when  we  have  conferences,  the 
students  and  I can  review  all  important 
information.  I will  explain  to  them  that 
they  have  access  to  the  folders  at  all 
other  class  times  during  the  month.  (If 
we  had  a copier  on  site,  I could  dupli- 
cate all  forms  so  students  could  keep 
one  and  I could  file  one.) 

Student  Joumals/Logs 

1 just  thought  I would  look  into  this  program 
to  what  it  was  all  about.  1 don't  know  what 
imll  come  of  this. .do  learn  for  me  has  always 
been  hard... 

Pat  (journal  entry,  7/91) 

Students  write  about  a wide  range  of 
topics  in  their  journals.  Some  write 
about  personal  issues  while  others 
reflect  on  their  learning  experiences  in 
class.  They  write  about  what  they 
understood  and  what  they  need  to 


practice.  They  also  comment  on  learning 
strategies  that  were  useful  and  those 
they  didn't  like.  Students  write  with  the 
understanding  that  I will  respond  to  the 
entry  but  will  not  correct  it.  They  write 
in  their  journals  at  the  end  of  every  class 
for  approximately  five  to  ten  minutes. 

Pat  is  now  working  on  her  fourth 
journal.  She  said  that  her  journal  pro- 
vides "good  (writing)  practice  and 
keeps  communication  open.  Students 
can  let  teachers  know  about  the  night' s 
class  and  what  was  interesting  or  hard." 

I agree  with  Pat's  assessment  of  the 
strengths  of  journals.  I gain  valuable 
information  from  reading  journal 
entries.  I learn  when  materials  or  learn- 
ing methods  were  useful  and  when 
students  feel  they  need  more  review.  I 
also  get  insight  into  the  personal  fives  of 
the  students  so  that  I better  understand 
their  goals,  interests,  and  difficulties. 

1 enjoyed  it  when  the  class  works  together... 

1 think  in  this  way  we  get  the  help  we 
need.. It  was  nice  to  see  my  writing  again.... 
I hope  others  enjoy  them  as  1 did  putting 
them  down  on  paper... 

Pat  (journal  entry,  5/11  /92) 

Teacher  Log/anecdotal  reporting 

We  reviewed  writing  folders  today  and  once 
again  1 was  troubled  by  their  use..some  use 
them  to  organize  and  set  goals. .ethers  have 
not  been  using  them  to  organize  their  drafts. 
I have  to  take  some  of  the  blame  because  we 
have  not  reviewed  them  for  a while..  As  I 
unite  this,  everyone  is  consulting  each  other 
for  spelling  and  word  usage  for  their  own 
journals.  This  group  helps  each  other  and 
relies  on  each  other  more  than  they  ask  me 
for  help. 

Eileen  (teacher  log,  5/24/93) 
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For  the  past  five  months.  I have  been 
keeping  a teacher  log.  I make  my  entries 
while  the  rest  of  the  class  write  in  their 
journals.  In  the  log,  I describe  the  day- 
to-day  events  of  the  class.  I record  GED 
test  scores,  breakthroughs  in  under- 
standing, and  comments  and 
suggestions  students  make  about  the 
class.  I also  try  to  record  anything 
students  tell  me  that  reflects  on  their 
progress  or  difficulties. 

When  I asked  Pat  whether  she 
thought  she  was  progressing,  she  re- 
sponded, "Sometimes.  I need 
confidence.  Whsn  I look  si  foldsrs  I fool 
confident  but  it's  different  going  for  the 
test"  She  is  very  concerned  she  will  not 
have  enough  time  to  complete  the  exam. 
She  did  feel  that  she  could  see  progress 
when  she  looked  at  the  papers  she  does 
in  class  and  when  she  corrects  them.  She 
added,  "I  still  have  a hard  time  with 
those  maps  and  charts!" 

Pat  reflected  on  specific  times  that 
she  knew  she  was  learning.  When 
remembering  the  time  spent  collaborat- 
ing with  others  to  write  the  environ- 
mental book,  she  said,  "It  was  nice  to 
work  as  a group  with  my  fellow  stu- 
dents. We  shared  ideas  and  were 
enthusiastic"  When  I asked  her  how  it 
felt  to  offer  a workshop  to  other  teachers 
and  students,  she  said,  "That  was  differ- 
ent! If  we  were  able  to  explain  to  others 
and  get  our  message  across,  it  means  we 
all  learned  together."  She  described 
having  her  work  published  in  Sharing 
Our  Thoughts  as  "...a  miracle!  I couldn't 
get  over  it.  Seeing  it  in  a book,  I must 
have  made  some  progress!" 

The  importance  of  recording  these 
types  of  comments  is  obvious  to  me. 

This  is  the  data  which  reflects  develop- 


ment of  self-esteem  while  also  noting 
hesitations  and  self-doubt.  It  is  my  hope 
that  by  recording  segments  of  conversa- 
tions with  students,  over  time,  I will 
gather  a true  reflection  of  their  views  of 
themselves  as  learners.  I recently 
learned  of  a teacher  who  leaves  her  log 
on  her  desk  for  students  to  read.  Learn- 
ers get  to  know  what  observations  the 
teacher  thinks  are  important  and  receive 
reinforcement  about  their  progress  and 
work  in  class.  I plan  to  make  my  log 
available  so  that,  just  as  with  the  other 
assessment  tools,  the  log  is  not  a device 
for  a teacher  to  m oasnre  a students 
progress,  but  a method  through  which 
learners  can  assess  their  own  growth. 

Community  Development 

When  examining  the  assessment 
tools  for  this  article  with  Pat,  I asked  her 
how  we  could  measure  community 
development.  She  became  highly  ani- 
mated and  said,  "You  have  to  have  more 
(than  studying  GED  subjects).  Young 
kids  just  don't  go  to  class  to  study  but  to 
talk  to  each  other,  express  opinions,  and 
have  a laugh  or  two...a  lot  have  left  but 
originally  we  had  a community  and 
friendship.  I was  going  through  some 
old  papers  the  other  day  and  thinking  of 
those  I knew  who  left  and  felt  so  blue 
and  really  empty  inside.  We  formed 
some  kind  of  friendship  with  people  we 
never  knew.  We  shared  times  trying  to 
learn.  We  said  our  opinions.  When 
something  was  new  or  hard  we  taught 
each  other.  Sometimes  we  laughed 
together  and  sometimes  we  got  upset 
with  each  other,  but  that's  how  it  was. 
We  shared  good  and  bad  times. ..It' s 
hard  that  the  group  is  moving  on  but  it 
made  me  think  that  I should  try  again." 
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Pat  highlighted 
the  value  of 
teaching  and 
learning  from  each 
other,  supporting 
each  other,  and 
developing  a sense 
of  responsibility 
towards  each 
other.  Her  sadness 
at  students  leav- 
ing and  the 
inspiration  she 
gained  from  their 
success  reinforced 
my  belief  in  the 
importance  of 
learners  returning 
to  the  class  as 
tutors  to  remain 
part  of  the  com- 
munity and  to 
serve  as  inspira- 
tion to  others. 


Pat's  views  confirmed  my  belief  in 
the  importance  of  developing  commu- 
nity. She  highlighted  the  value  of 
teaching  and  learning  from  each  other, 
supporting  each  other,  and  developing  a 
sense  of  responsibility  towards  each 
other.  Her  sadness  at  students  leaving 
and  the  inspiration  she  gained  from 
their  success  reinforced  my  belief  in  the 
importance  of  learners  returning  to  the 
class  as  tutors  to  remain  part  of  the 
community  and  to  serve  as  inspiration 
to  others.  I also  realized  that  Pat  uses 
language  that  expresses  a sense  of 
commitment  to  the  groups  and  the 
project  such  as  "fellow  students"  and 
"taught  each  other."  In  the  future,  it  will 
be  important  to  record  this  language 
and  to  trace,  over  time,  how  and  when 
learners  use  it.  Mosdy,  however,  I still 
feel  frustrated  by  my  limited  attempts  to 
measure  what  I believe  is  the  key  to 
learning.  I will  continue  to  look  to 
learners  and  other  colleagues  for  ways 
in  which  to  measure  this  progress. 

I am  grateful  to  Pat  for  agreeing  to 
discuss  these  issues  with  me  and  to 
collaborate  on  this  piece.  I am  in  the 
process  of  discussing  similar  questions 


with  the  rest  of  the  group.  What  I 
learned  from  our  discussions  is  that  we 
need  to  talk  about  and  reflect  more  on 
assessment  in  class.  We  need  to  have 
more  frequent  conversations  about 
learning  moments  and  more  time  to- 
gether to  analyze  the  data  gained  from 
these  checklists.  The  danger  of  using 
only  teacher-generated  checklists  has 
become  more  clear  to  me.  I am  still 
uncertain  about  the  best  ways  to  docu- 
ment the  development  of  self-esteem 
and  community,  but  I do  feel  that  my 
teacher  log  will  play  an  important  part. 
Most  importantly,  I will  ask  learners  for 
feedback  about  the  tools  we  use.  Their 
insights  help  me  to  realize  just  who  this 
information  is  for  and  for  what  pur- 
poses. 

When  I asked  Pat  if  there  was  any- 
thing I could  do  to  motivate  her  to 
continue  to  come  to  class  and  to  feel 
confident  enough  to  take  the  GED  tests, 
she  said,  pointing  to  her  heart,  "No,  it 
has  to  come  from  in  here."  I believe  this 
most  eloquently  expresses  the  need  for 
learners  to  be  in  charge  of  assessing 
themselves  and  monitoring  their  own 
progress. 
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READING  PROGRESS  CHECKUST 

Date: 


Usually  Sometimes  Rarely  Not  Yet 


1.  I understand  what  I read  in  class 

2.  I understand  what  I read  outside  class. 

3.  I talk  about  what  I read  in  class. 

4.  I think  about  what  I already  know  before  I read. 

5.  I think  about  what  I already  know  as  I read. 

6.  I ask  myself  questions  as  I read. 

7.  In  my  own  mind,  I say  what  I read  in  my  own 
words 

8.  If  I don't  know  a word,  I use  the  words  around 
it  to  guess  what  it  means. 

9.  If  i don't  know  a word,  I think  about  how  the 
letters  sound  and  "sound  it  out." 

1 0. 1 slow  down  if  the  reading  is  hard  or  spped  up  If 
the  reading  is  easy. 

1 1 . After  I read,  I think  about  what  I read  and  think 
if  I agree  or  disagree. 

How  has  your  reading  improved? 


What  do  you  need  to  work  on? 
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Date;  30~  93 


Usually 

Sometimes 

Rarely  f 

Not  Yet 

1 . 1 understand  what  1 read  tn  class 

2.  I understand  what  I read  outside  class. 

3.  ! talk  about  what  I read  In  class. 

y 

4.  ! think  about  what  1 already  know  before  1 read. 

s/ 

5.  1 think  about  what  I already  know  as  I read. 

6.  1 ask  myself  questions  as  I reaH. 

7.  In  my  own  mind,  1 say  what  1 read  in  my  own 
words 

8.  If  1 don't  know  a word,  I use  the  words  around 
it  to  guess  what  it  means. 

9.  If  I don't  know  a word,  1 think  about  how  the 
letters  sound  and  “sound  it  out.” 

V' 

V" 

1 0. 1 slow  down  if  the  reading  is  hard  or  spped  up  if 
the  reading  is  easy. 

v/ 

1 1 . After  1 read,  1 think  about  what  1 read  and  think 
if  1 agree  or  disagree. 

How  has  your  reading  improved? 

What  do  you  need  to  work  on? 
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Reader  •sJL  — Date: 


■ - - 

usuaUy 

-somei. 

times 

prely. 

not.. 

yet 

1.  1 understand  what  1 read  In  class. 

s . 

.2.  1 understand  what  1 read  outside 

7 

oPcTass. 

r 

3.  1 talk  about  what  1 read  In  class. 

7 

4.  1 thlnjc  about  what  1 already  know 
before  1 read. 

5.  i think  about  what  ! already  knew 
as  l read. 

V 

1 

i 

i 

i 

6.  1 ask  myself  questions  as  1 read. 

! 

7.  in  my  mind,  1 say  what  1 read  In 
my  own  words. 

8.4fHHlon't  know  a word,  I use-the-’  • 
words  around  it  to  guess  what  It  means. 

9.  If  1 don’t  know  a word,  1 think  about 
how  the  letters  sound  and  ‘sound  It  out.* 

^ 

10.  1 slow  down  If  the  reading  is  hard  or 
speed  up  If  the  reading  Is  easy. 

l/ 

/#.  After  l read,  1 think  about  what  1 read 
and  think  If  1 agree  of  disagree. 

7 

i 

! 

How  has  your  reading  improved? 


What  do  you  need  to  work  on? 
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Book  Review 

It  Belongs  to  Me:  A Guide  to  Portfolio 
Assessment  in  Adnlt  Education  Programs 


It  Belongs  to  Me:  A Guide  to  Portfolio 
Assessment  in  Adult  Education  Pro- 
grams, by  Hanna  Arlene  Fingeret 

For  educators  at  all  levels  who  are 
dissatisfied  with  standardized  tests 
as  the  primary  or  sole  means  of  student 
evaluation,  and  who  are  exploring 
various  paths  in  what's  come  to  be 
called  "alternative  assessment,"  the  use 
of  portfolios  and  portfolio  assessment  is 
one  of  the  major  options.  Yet  the  concept 
of  portfolios  is  not  always  clearly  under- 
stood, and  many  practitioners  who  are 
interested  may  have  questions  about 
how  to  implement  this  approach. 

Hanna  Arlene  Fingeret,  a well- 
known  adult  educator  who  works  with 
the  Literacy  South  organization  in 
Durham,  North  Carolina,  was  recently 
commissioned  by  the  US.  Department 
of  Education  to  conduct  a study  of 
portfolio  assessment  in  the  world  of 
adult  literacy.  It  Belongs  to  Me:  A Guide  to 
Portfolio  Assessment  in  Adult  Education 
Programs,  the  result  of  this  study,  is 
"designed  to  introduce  adult  literacy 
educators  to  the  concept  of  portfolio 
assessment,  and  to  provide  some  guid- 
ance about  how  you  can  incorporate 
portfolio  assessment  into  your  work  in 
adult  literacy  education."  It  provides  a 
clear,  succinct,  and  very  useful  introduc- 
tion. 

Fingeret  conducted  both  an  extensive 
literature  review  and  a wide-ranging 
series  of  interviews  with  more  than  50 


students,  teachers,  administrators,  staff 
developers,  and  other  practitioners 
across  the  country  who  are  using  portfo- 
lios in  their  adult  literacy  teaching  and 
learning.  Quotations  from  these  inter- 
views pop  up  throughout  the  text  and 
constitute  an  important  feature  of  this 
guide — important  because  so  much  of 
fVig  literature  on  portfolios  b?s  omsr^cd 
from  the  K-12  or  college  contexts,  while 
these  quotes  present  voices  from  the 
field  of  adult  literacy;  important,  too, 
because  they  ground  the  discussion  in 
reality  in  ways  that  no  summary  of 
findings  by  an  author  can  ever  really  do; 
and  important  because,  while  acknowl- 
edging the  difficulties  of  implementing 
portfolio  assessment,  these  practitioners 
and  learners  dearly  demonstrate  their 
enthusiasm  for  the  use  of  portfolios  and 
their  words  will  encourage  others  to 
move  forward  into  this  area  as  welL 
It  Belongs  to  Me  begins  with  a brief 
discussion  of  what  is  meant  by  the  term 
"portfolio  assessment."  A portfolio  is 
defined  here  as  a selection  of  various 
materials  that  have  been  chosen  from 
other,  larger,  on-going  collections  of 
materials  related  to  a student's  work 
and  achievement,  both  inside  and 
outside  the  classroom.  All  sorts  of  items 
could  be  induded  in  portfolios — pieces 
of  writing,  written  responses  to  reading, 
reading  logs,  daily  work,  checklists, 
special  projects,  to  name  a few  of  the 
possibilities.  The  distinction  between  the 
portfolio  itself  and  the  folders  or  other 
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The  process  by 
which  students 
review  their  work, 
select  certain 
items  for  inclusion 
in  the  portfolio, 
and  then  assess 
this  collection  is 
probably  the 
crucial  aspect 
of  the  whole 
approach  as 
Fingeret 
describes  it. 


"containers"  from  which  materials  are 
chosen  for  the  portfolio  is  an  important 
one  and  is  not  merely  a matter  of  seman- 
tics or  simply  an  issue  of  quantity.  The 
process  by  which  students  review  their 
work,  select  certain  items  for  inclusion 
in  the  portfolio,  and  then  assess  this 
collection  is  probably  the  crucial  aspect 
of  the  whole  approach  as  Fingeret 
describes  it  The  value  of  using  portfolio 
assessment  lies  not  just  in  its  passive 
product — the  collection  of  items  repre- 
senting various  aspects  of  a student's 
work  and  achievement — but  also  in  its 
active  process,  as  a student  gains  valu- 
able understanding  of  his/her  own 
learning  and  accomplishments  through 
selecting  material  for  his/her  portfolio 
and  then  assessing  this  portfolio  collec- 
tion itself.  As  writers  Zessoules  and 
Gardner  note  in  an  article  on  alternative 
assessment  quoted  by  Fingeret,  "No 
longer  a weapon  for  rooting  out  and 
combating  students'  weaknesses,  assess- 
ment becomes  an  additional  occasion  for 
learning." 

Having  looked  at  what  portfolio 
assessment  is  (and  isn't),  Fingeret  then 
takes  readers  through  a cyclical,  four- 
stage  implementation  process  organized 
under  the  headings  of  1)  choosing,  2) 
planning,  3)  implementing,  and  4) 
evaluating  and  revising  portfolio  assess- 
ment. Stage  One  asks  teachers  who  are 
thinking  of  trying  portfolio  assessment 
to  begin  by  deciding  whether  portfolio 
assessment  is  consistent  with  their  views 
of  literacy,  instruction,  and  assessment. 
Fingeret  reviews  the  various  ways  in 
which  literacy  is  seen  in  the  US.  at  this 
time,  the  different  approaches  to  adult 
literacy  instruction  that  exist  within  the 
field,  and  the  current  issues  surrounding 


the  use  of  standardized  tests  vs.  various 
types  of  alternative  assessment  in  adult 
literacy.  She  notes  that  "portfolio  assess- 
ment is  compatible  with  instruction  that 
approaches  literacy  as  a process  of 
constructing  meaning,  in  a learner- 
centered  way,"  but  that  "if  you  find  that 
you  support  a more  skills-based  view  of 
literacy,  or  that  you  see  assessment  as 
dependent  on  standardized  test  scores 
and  the  judgments  of  outside  experts, 
then  portfolio  assessment  will  not  be 
appropriate  for  your  classroom  or 
program." 

Teachers  who  elect  to  embark  on  a 
voyage  of  portfolio  assessment  are  then 
shown  in  Stage  Two  a five-step  planning 
process  in  which  they  begin  to  make 
decisions  about  and  start  to  develop: 

4 the  focus  for  the  portfolio  assessment 
process  (for  example,  writing,  reading, 
math,  everything)  and  how  pertinent 
materials  will  be  collected  in  an  on- 
going way  in  folders  or  other  containers; 
4 the  schedule  by  which  students  will 
review  the  material  in  their  folders  and 
develop  their  portfolios; 

4 the  criteria  to  be  used  by  students  in 
selecting  material  for  the  portfolios; 

4 the  process  students  will  use  in 
making  choices  and  moving  materials 
from  folder  to  portfolio;  and 
4 the  criteria  and  process  for  assessing 
the  contents  of  the  portfolios  on  a peri- 
odic basis.  Teachers  are  urged  to  work 
with  and  get  support  from  other  teach- 
ers while  involved  in  this  planning 
process  (as  well  as  during  the  other 
stages  of  portfolio  assessment). 

As  one  teacher  says,  "The  fact  that  I 
met  with  four  other  staff  members  on  a 
regular  basis  was  extremely  helpful.  It 
kept  us  motivated  and  kept  the  momen- 


O ’ENTIRES  IN  ASSESSMENT 


412 


Volume  5:  Fall  1999 


turn  going  of  the  pilot  project...  Support 
is  really  important  when  you're  piloting 
portfolio  assessment"  You  can  make 
this  exploration  a solo  flight,  but  it's 
better  to  go  with  others. 

Stage  Three  focuses  on  implementing 
portfolio  assessment  by  following 
through  on  all  the  planning  that  was 
done  in  Stage  Two.  First,  introduce  the 
concept  of  portfolio  assessment  to 
students  and  encourage  them  to  begin  to 
think  of  portfolios  as  "an  integral  part  of 
instruction."  Students  then  continue  on 
with  this  process— creating  folders  to 
rnllprt  materials,  develnnimr  rritena  fnr 

JT  O 

choosing  materials  to  move  from  the 
folders  to  their  portfolios,  actually 
reviewing  materials  and  making  selec- 
tions, and  assessing  the  portfolios  of 
assembled  materials.  Though  usually 
based  on  the  general  guidelines  devel- 
oped by  the  teacher  during  the  planning 
stage,  the  specifics  of  the  portfolio 
assessment  process  will  emerge  out  of  a 
(probably  on-going)  process  of  reflec- 
tion, negotiation,  and  decision-making 
by  the  students  themselves,  both  indi- 
vidually and  as  a class. 

Stage  Four  calls  for  teachers  to 
evaluate  the  portfolio  assessment  pro- 
cess they  have  tried  out  and  to  make 
changes  as  needed.  Since  there  is  no 
perfect  recipe  for  carrying  out  portfolio 
assessment  and  since  it  is  seen  as  a 
cyclical  process  of  reflection,  planning, 
implementation,  and  evaluation,  teach- 
ers may  want  to  circle  back  to  any  of  the 
previous  stages  before  continuing  on. 

It  Belongs  to  Me  ends  by  stepping 
back  to  look  at  the  impact  of  portfolio 
assessment  on  students  and  teachers,  at 
the  process  of  implementing  portfolio 
assessment  at  the  program  level,  and  at 


what  steps  are  needed  to  support  the 
continued  growth  of  portfolio  assess- 
ment in  adult  literacy.  There  are  also  an 
appendix  containing  a few  sample 
documents  used  by  some  programs  in 
their  portfolio  assessment  process  and 
an  annotated  bibliography  of  materials 
dealing  with  portfolio  assessment. 

Part  of  the  on-going  message  of  this 
guide  is  that  doing  portfolio  assessment 
isn't  easy.  It's  usually  not  something 
that  either  teachers  or  students  are 
accustomed  to  doing,  and  getting  com- 
fortable with  the  process  can  take  some 
time.  Gathering  materials  for  folders  is 
much  easier  in  some  areas  (such  as 
writing)  than  in  others  (such  as  oral 
language  use).  Various  cultures  may 
vary  in  how  they  regard  the  idea  of 
individuals  highlighting  their  own 
achievements.  Reporting  the  results  of 
portfolio  assessment  to  funders,  evalua- 
tors, and  policy  makers  is  not  a simple 
process.  Working  conditions  in  the  field 
are  often  not  conducive  to  doing  portfo- 
lio assessment,  which  requires  sufficient 
time  (for  staff  development,  planning, 
and  preparation)  and  sufficient  space 
(for  secure  storage  of  folders  and  portfo- 
lios). Fingeret  and  her  "commentators" 
do  address  all  these  issues.  Yet,  without 
minimising  the  difficulties,  they  also 
make  clear  that  doing  portfolio  assess- 
ment is  worth  it. 

Summing  up  its  impact,  Fingeret 
says,  "Portfolio  assessment  is  practical 
and  useful,  according  to  the  practitio- 
ners and  students  who  participated  in 
this  project  It  redefines  the  scope  of 
assessment,  and  provides  a way  to  look 
at  personal  development  as  well  as 
academic  skill  growth  as  reflected  in 
new  literacy  practices.  It  facilitates  a 
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With  theportfolio 
the  students  saw 
exactly  where  they 
improved,  they 
saw  the  kind  of 
gain.  People  still 
talk  to  me  about 
that....  They 
definitely  want  to 
continue  with 
portfolio  assess- 
ment. 


deeper  level  of  reflection  for  students, 
and  a deeper  level  of  communication 
between  students  and  teachers.  It  also 
promotes  professional  development  and 
practitioner  inquiry." 

Here  are  a few  brief  quotes  from 
some  of  the  teachers,  administrators, 
and  staff  developers  who  contributed  to 
this  project 

In  all  my  experiences,  students  are  really 
excited  about  portfolio  assessment  and  they 
feel,  'This  makes  sense.' 

It's  empowering  to  students,  and  that's  what 
I emphasize,  how  a portfolio  helps  students 
become  independent  learners  and  take 
responsibility  for  assessing  their  own  work. 

Portfolio  assessment  has  minimized  student 
attrition,  and  it  seemed  to  keep  staff  better 
too.  Portfolios  help  adults  see  where  they're 
at  and  therefore  they  stay....  Portfolios  value 
teacher  judgment;  by  giving  multiple  choice 
standardized  tests,  the  message  is  that 
nobody's  judgment  counts  except  for  the  test 
publisher. 

When  we  did  the  share  last  year  and  people 
shared  their  portfolios  there  was  a lot  of 
pride....  With  the  portfolio  the  students  saw 
exactly  where  they  improved,  they  saw  the 
kind  of  gain.  People  still  talk  to  me  about 

that They  definitely  want  to  continue  with 

portfolio  assessment. 


Students,  too,  had  positive  things  to 
say  about  the  process.  Loma  Irizarry,  for 
example,  is  a student  in  New  York,  and 
it  is  from  her  words  that  the  title  for  this 
report  was  taken.  She  says,  "What  I 
learned  from  my  portfolio  is  that  I've 
achieved  what  I wanted.  I feel  relaxed.  It 
belongs  to  me.  It's  for  me  to  know,"  and 
"Seeing  my  progress  in  the  portfolio 
makes  me  know  I can  do  the  work." 

There  is,  of  course,  no  "one  size  fits 
all"  set  of  specific  instructions  for  how  to 
set  up  and  do  portfolio  assessment  How 
it  works  out  for  one  program,  one 
teacher,  one  group  of  students  will  vary 
a great  deal  from  what  happens  in  other 
places,  at  other  times,  with  other  people. 
And  it's  also  important  to  remember 
that  portfolio  assessment  isn't  the  only 
possible  approach  to  take  when  embark- 
ing upon  alternative  assessment;  many 
other  ideas  have  been  presented  here 
in  the  various  issues  of  Adventures  in 
Assessment  and  elsewhere.  Fingeret's 
guide,  however,  should  provide  a very 
helpful  framework  for  teachers  inter- 
ested in  trying  out  this  particular 
approach. 

Copies  of  It  Belongs  to  Me  can  be 
obtained  free  of  charge  by  writing  to  the 
Clearinghouse,  Division  of  Adult  Educa- 
tion and  literacy,  U5.  Department  of 
Education,  Washington,  DC  20202-7240; 
FAX  (202)  205-8973.) 
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Letter  from  the  Field 

The  Case  for  Pre-Goal  Setting 


Goal  setting  is  a linear,  future- 

oriented,  individualistic.  Western 
phenomenon  which  is  not  always 
transferable  to  English  as  a Second 
Language  (ESL)  learners  from  non- 
Wes  tern  cultures  and  other  domestic 
cultural  contexts 

Yet  many  educators  in  our  field 
propose  goal  setting  as  the  panacea  to 
all  learners'  problems.  Goal  setting  as  an 
activity  has  been  used  in  programs  for  a 
long  time  yet  retention  rates  are  still 
low.  In  this  paper  I propose  a "pre-goal- 
setting" strategy  as  a panacea  to  some  of 
the  ills  that  goal  setting  activities  do  not 
address. 

GOAL  SETTING  AS  A 
MIDDLE  CLASS  PHENOMENON 
I spent  a number  of  years  working 
overseas  as  an  educator  and  also  work- 
ing as  an  educator  with  rural 
immigrants,  refugees  and  Adult  Lit- 
eracy/Adult Basic  Education  (AL/ ABE) 
learners  in  the  United  States. 

I was  bom  in  a housing  project,  spent 
most  of  my  life  in  the  United  States 
living  in  the  inner  city  and  avoided 
schooling  as  much  as  possible  up  to  the 
age  of  sixteen.  Based  on  those  "growing 
experiences"  (not  negative  experiences, 
as  so  often  perceived  and  referred  to  by 
some  adult  education  professionals)  and 
from  that  perspective,  I view  goal 
setting  as  an  strategy  for  motivating 
learning  that  is  embedded  with  middle 
class  values. 


On  its  own,  goal  setting  may  not 
work  well  with  many  ESL  learners  who 
are  economically  disadvantaged,  nor 
with  individuals  who  grew  up  in  an 
environment  where  it  was  not  part  of 
their  value  system.  Goal  setting  is  often 
taken  out  of  context  and  transferred  to 
individuals  who  may  never  have  seen 
any  COnCT6tS  rPwarHc  fnr  ♦Jroir  Aum 

individual  efforts  or  the  efforts  of 
groups  to  which  they  belong. 

As  a Western,  middle-class  activity,  it 
works  in  that  context  primarily  because 
there  are  numerous  visible  rewards. 
When  people  without  positive  role 
models,  mentors  or  solidarity  in  a group 
(gang)  do  set  goals,  it  is  easier  for  them 
to  "fall  short"  than  to  follow  through 
with  the  necessary  steps  needed  to 
achieve  them. 

"Success"  in  Adult  Literacy/ Adult 
Basic  Education  (AL/ABE)  doesn't 
necessarily  start  with  adults  walking 
into  a program  and  setting  goals.  Nor 
does  it  start  with  teachers  expecting 
adult  learners  to  be  able  to  immediately 
be  successful  at  using  goal-setting. 

Teachers  should  not  assume  that 
people  with  very  different  life  experi- 
ences will  proceed  through  classes  as 
they  themselves  probably  proceeded 
and  succeeded  through  schooling  and 
life. 


by 

Don  Robishaw 


Evaluation  and 
Research  Consultant 
SABES  Central 
Resources  Center 
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I agree  that  goal 
setting  as  a 
strategy  can  help 
some  learners  take 
control  of  their 
own  learning  and 
become  motivated. 
But  "pre-goal- 
setting” strategies 
must  be  designed 
and  implemented 
to  better  help 
adults  develop 
confidence  in  their 
own  ability  to  use 
goal  setting  as  a 
strategy  for 
learning. 


A CASE  FOR  "PRE-GOAL-SETTING" 
AS  PART  OF  A SELF-DIRECTED 
LEARNING  STRATEGY 

Adults  in  their  everyday  lives  are 
motivated  to  leam  many  things.  Some 
day  it  will  become  necessary  for  them  to 
leam  on  their  own  in  class.  Learning  on 
their  own  is  important  because  many 
learners  fear  they  cannot  leam  without 
the  aid  of  a teacher  or,  even  worse,  that 
they  can  not  leam  at  all.  They  must 
overcome  this  anxiety  and  develop  self- 
confidence.  Responsible  adults  need  to 
take  control  of  their  own  learning. 

At  the  Read/Write/Now  Program  in 
Springfield  Janet  Kelly  uses  goal-setting 
as  a small  group  learning  strategy  for 
learners  with  similar  goals,  such  as 
obtaining  a driver's  license.  These  goals 
are  identified  from  the  student's  goals 
list 

I agree  that  goal  setting  as  a strategy 
can  help  some  learners  take  control  of 
their  own  learning  and  become  moti- 
vated. But  "pre-goal-setting"  strategies 
must  be  designed  and  implemented  to 
better  help  adults  develop  confidence  in 
their  own  ability  to  use  goal  setting  as  a 
strategy  for  learning.  Qt  is  important  to 
remember  that  these  are  strategies  rather 
than  activities,  because  the  teachers 
must  continue  to  pay  close  attention  to 
the  learners'  important  emotional  and 
psychological  processes  as  well  as 
intellectual  processes.) 

Many  learners  need  to  participate  in 
a "process"  in  order  to  find  goal  setting 
acceptable.  Later  I will  recommend  a 
strategy  that  may  increase  the  chances  of 
goal  setting  being  more  effective.  But 
first  it  is  important  to  understand  that 
goal  setting,  as  a strategy,  is  just  a small 
step  in  one  part  of  a whole  Adult  Educa- 


tion philosophy  which  is  self-directed 
learning. 

Self-directed  learning  can  be  divided 
into  external  and  internal  processes. 
Malcolm  Knowles  is  perhaps  best 
known  for  his  understanding  of  the 
external  processes: 

. . .a  process  in  which  individuals  take 
the  initiative,  loith  or  without  the  help  of 
others,  in  diagnosing  their  learning  needs, 
formulating  learning  goals,  identifying 
human  and  material  resources  for  learning, 
choosing  and  implementing  appropriate 
learning  strategies,  and  evaluating  learning 
outcomes.  (Knowles,  M.  1975,  p.  18). 

The  criteria  assume  that  a learner 
need  only  have  the  ability  to  leam  on 
his/her  own  and  to  manage  instruc- 
tional activities.  Educators  who  try  to 
instill  these  skills  in  their  learners  may 
be  assuming  that  the  learners  are  aware 
and  consciously  prepared  to  accept 
them. 

Another  piece  has  often  been  either 
neglected  or  assumed  in  AL/ABE.  This 
other  side  involves  the  internal  changes 
needed  to  become,  as  Brookfield  would 
point  out,  fully  self-directed. 

...self-directed  learning  is  concerned 
much  more  with  an  internal  change  of 
consciousness  than  with  the  external  man- 
agement of  instructional  events.  This 
consciousness  involves  an  appreciation  of 
the  contextuality  of  knowledge  and  an 
awareness  of  the  culturally  constructed  form 
of  value  frameworks,  belief  systems  and 
moral  codes  that  influence  behavior  and  the 
creation  of  social  structures.  (Brookfield,  S., 
1985,  p.  15) 

Adult  learners  need  to  reflect  on  their 
past  life  history  and  to  sort  through  their 
experiences  in  order  to  see  beyond  their 
own  formal  schooling  experiences  in 
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order  to  better  develop,  persist  and 
continue  with  learning. 

Some  adult  learners  need  to  channel 
their  former  resistance  to  schooling  into 
the  persistence  needed  to  become  self- 
directed  learners.  A strategy  might  be  to 
allow  learners  to  reflect  on  how  formal 
schooling  made  it  more  difficult  for 
them  as  youths  to  accept  the  dominant 
cultural  value  system  because  of  their 
lack  of  dominant  "cultural  capital" 
(assets)  that  their  more  well-off  school- 
mates had  developed  prior  to  beginning 
school.  Rather  than  ameliorating  this 
problem,  schools  did  much  to  perpetu- 
ate the  differences.  Many  learners 
resisted  the  dominant  value  system 
advocated  by  the  schools. 

After  developing  persistence  they 
may  then  be  able  to  make  the  behavioral 
changes  necessary  to  accept  goal  setting 
strategies  and  even  more  important, 
self-directed  learning.  Learners  need  to 
discover  alternatives  through  a "dialogi- 
cal process"  with  their  peers. 

ACTIVITIES  VS.  STRATEGIES 

This  is  one  strategy  in  addition  to 
other  strategies  mentioned  in  Group  Goal 
Setting  Activities:  An  Approach  from  Youth 
Services  Corps,  an  article  that  appeared  in 
the  previous  edition  of  Adventures  in 
Assessment.  The  Youth  Services  Corps 
first  suggested  "activity"  is  a participa- 
tory group  process  that  allows  learners 
opportunities  to  self-reflect  and  to  share 
their  experiences  through  a dialogical 
pedagogy  that  respects  learners  as 
having  valuable  prior  life  experiences. 
Their  first  suggested  "activity"  is  called 
"Thinking  about  Learning". 

Their  process  begins  with  participa- 
tion. I know  little  about  the  organization 


nor  their  retention  rates.  Although  their 
"activities"  may  not  be  designed  exclu- 
sively to  prevent  dropouts,  I am  curious 
to  know  if  these  suggested  "activities"  at 
all  contribute  to  reducing  their  drop  out 
rates. 

My  only  problem  is  they  refer  to 
what  they  do  as  activities  rather  than 
strategies.  There  is  a difference  between 
goals,  which  should  be  addressed 
through  a strategy  and  competencies, 
such  as  found  on  some  reading  skills 
and  life  skills  checklists,  which  should 
be  addressed  through  activities.  (See 
Janet  Kelly's  article  in  the  first  issue  of 
Adventures  in  Assessment)  Maybe  this 
is  only  a semantics  issue,  but  I hope  in 
the  future  that  they  choose  to  call  the 
wonderful  things  they  do  strategies 
rather  than  activities.  Activities  are  fine 
for  addressing  competencies  but  it  is 
better  to  choose  strategies  to  address 
goals. 

BEGINNING  WITH 
THE  INTERNAL  PIECE 

I also  suggest  a small  group  dialogi- 
cal strategy  to  allow  learners  the 
opportunity  to  self-reflect  and  to  share 
their  reflections  and  experiences  with 
their  own  cohorts.  Dialogical  pedagogy 
is  participatory  and  respects  the  learners 
as  adults  with  numerous  valuable  life 
experiences  or,  as  Shor  points  out:  "A 
participatory  class  begins  with  participa- 
tion. A critical  and  empowering  class 
begins  by  examining  its  subject  matter 
from  the  students  point  of  view  and  by 
helping  students  see  themselves  as 
knowledgeable  people  (Shor,  L,  1992,  p. 
37).  The  facilitator  creates  an  environ- 
ment that  respects  the  learners  and 
allows  the  learners  the  freedom  and 
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opportunity  to  grow  and  develop. 

But  what  is  often  neglected  by  facili- 
tators who  use  a dialogical  pedagogy 
with  AL/ ABE  learners  is  the  need  to 
first  set  and  establish  the  ideal  condi- 
tions for  discourse  before  entering  into  a 
group  discussion. 

Another  important  aspect  often 
neglected  is  the  need  for  solidarity 
building  between  the  participants  and 
the  facilitator.  At  the  Read/Write/Now 
program. 

It  is  all  part  of  the  activities  early  in  the 
class  cycle.  We  usually  have  a group  lesson / 
discussion  on  goal  setting.  We  have  also 
used  readings  from  leamers-xoritten  publica- 
tions or  oral  histories  which  touch  on 
educational  experiences  and  life  experiences 
that  many  learners  can  identify  zoith  as  a 
starting  place  for  analysis  and  discussion 
about  where  we  have  been,  where  we  want  to 
go,  and  how  we  can  gel  there " (Kelly,  J 
1991,  P.24). 

Janet  does  a lot  more  at  the  Read/ 
Write/Now  Program  in  this  area  than 
most  programs  and  I applaud  her  work 
and  the  same  goes  for  the  Youth  Service 
Corps  Program  in  Philadelphia. 

Yet  when  it  comes  to  developing 
solidarity  it  is  very  important  that  the 
facilitator  has  had  similar  life  experi- 
ences and  experiences  with  formal 
schooling  as  the  learners.  I can  read, 
study  and  talk  to  a lot  of  people  about 
breast  cancer,  battered  women,  alcohol- 
ism... but  that  doesn't  qualify  me  to 

facilitate  a discussion  group  made  up  of 
these  victims.  It  is  no  different  with 
Adult  Literacy  and  Adult  Basic  Educa- 
tion learners. 


CONCLUSION 

Adult  learners  can  benefit  from 
programs  which  center  on  pedagogy 
that  focuses  on  participation,  dialogue, 
and  self-reflection  prior  to  engaging  in 
goal-setting  strategies  or  other  aspects  of 
external  self-directed  learning.  In  addi- 
tion to  increasing  the  chances  that  goal 
setting  will  work,  the  process  will  also 
help  empower  both  learners  and  facilita- 
tors. Self-reflection  is  an  important  piece 
in  Adult  Education  and  teachers  will 
also  learn  much  about  themselves  and 
their  practice  through  self-reflection. 
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System  tor  AduS  Basic  Education  Supoort 


SABES  is  the  System  for  Adult  Basic  Education  Support ; a comprehensive  training  and  technical 
assistance  initiative  for  adult  literacy  educators  and  programs.  Its  goal  is  to  strengthen  and  enhance 
literacy  services  and  thus  to  enable  adult  learners  to  attain  literacy  skills.  SABES  accomplishes  this 
goal  through  staff  and  program  development  workshops , consultation , mini-courses,  mentoring  and 
peer  coaching,  and  other  training  activities  provided  by  five  Regional  Support  Centers  located  at 
community  colleges  throughout  Massachusetts. 

SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect,  evaluate,  and  disseminate  ABE 
materials,  curricula,  methodologies,  and  program  models.  Each  of  the  five  SABES  Regional  Support 
Centers  similarly  offers  program  support  and  a lending  library.  SABES  maintains  an  Adult  Literacy 
Hotline,  a statewide  referral  service  which  responds  to  calls  from  new  learners  and  volunteers.  The 
Hotline  number  is  1-800-447-8844. 

The  SABES  Central  Resource  Center,  located  at  World  Education,  publishes  a statewide  quarterly 
newsletter,  “Bright  Ideas,  99  and  journals  on  topics  of  interest  to  adult  literacy  professionals,  such  as 
this  volume  of  “Adventures  in  Assessment. 99 

The  first  three  volumes  of  “Adventures  in  Assessment  99  present  a comprehensive  view  of  the  state  of 
practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners.  Volume  1,  Getting 
Started,  included  start-up  and  intake  activities.  Volume  2,  Ongoing,  shares  tools  for  ongoing 
assessment  as  part  of  the  learning  process;  Volume  3,  Looking  Back,  Starting  Again,  focused  on  tools 
and  procedures  used  at  the  end  of  a cycle  or  term,  including  self  class,  and  group  evaluation  by  both 
teachers  and  learners.  Volume  4 covered  a range  of  interests.  Volume  5,  Tale  of  the  Tools  was 
dedicated  to  reflecting  on  component  3 tools  of  alternative  assessment.  Volume  6,  Responding  to  the 
Dream  Conference,  was  dedicated  to  responses  to  Volumes  1-5.  Volume  7,  The  Partnership  Project, 
highlighted  writings  from  a mentoring  project  for  practitioners  interested  in  learning  about 
participatory  assessment.  Volume  8 covered  a range  of  topics  from  education  reform  to  learner 
involvement  in  assessment.  Volume  9 looked  at  assessment  in  a volunteer  program,  native  language 
literacy,  ESOL  programs,  and  evaluations  in  workplace  education  programs. 

We  would  like  to  see  your  contribution , especially  if  you  are  doing  native  language  literacy  or  low- 
level  ESOL  literacy.  Contact  editor,  Alison  Simmons,  to  discuss  your  submission. 

Permission  is  granted  to  reproduce  portions  of  this  journal.  We  request  appropriate  credit  be  given 
to  “Adventures  in  Assessment 99  and  to  the  author. 

“Adventures  in  Assessment”  is  free  to  DOE-fimded  Massachusetts  programs;  out-of-state  requests 
will  be  charged  a nominal  fee,  at  cost.  Please  write  to,  or  call: 

Alison  Simmons 
SABES  Central  Resource  Center 
World  Education 
44  Farnsworth  Street 
Boston,  MA  02210-1211 
617-482-9485 
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Final  Reflections  from  Your  Editor 

Creating,  Editing,  and  Producing 
the  Dream  Conference 


by 

Loren  McGrail 


Editor 

SABES  Central 
Resource  Center/ 
World  Education 


As  a sometime  adult  literacy  practitioner, 
particularly  interested  in  alternative  assess- 
ment, reading  through  all  five  issues  of 
Adventures  in  Assessment  felt  akin  to 
attending  a dream  conference. 

Cathy  Luna 

It  is  spring  and  it  is  still  snowing. 

The  crocuses  are  shivering  but 
Passover  is  here  and  Easter  is 
around  the  comer.  Getting  this  journal 
out  the  door  and  into  the  anonymous 
hands  of  practitioners  around  this  state 
and  all  over  the  country  feels  a little 
more  like  a nightmare  than  a dream 
during  this  phase  of  the  production  cycle 
but  the  dream  is  still  alive:  the  simple 
belief  that  a journal  dedicated  to  writ- 
ings on  learner-centered  or  participatory 
approaches  to  assessment  and  evalua- 
tion written  by  and  for  practitioners  will 
be  of  use. 

This  issue  of  Adventures  in  Assessment 
is  the  last  issue  that  I will  edit.  The 
journal  will  continue  to  be  published  bi- 
annually  by  SABES.  It  is  an  issue  prima- 
rily devoted  to  responses  to  the  journal, 
all  5 volumes.  The  field  is  vast  and  wide. 
The  authors  in  volume  6 include  a 
doctoral  candidate  in  a graduate  school 
of  education,  an  ESOL  teacher  in  a 
community  based  program,  a staff 
development  facilitator,  a tutor  trainer,  a 
state  ABE  director  and  the  former  assis- 
tant director  of  a national  clearinghouse. 

I invite  you  to  listen  and  drink  in  their 
thoughts  and  reflections,  how  and  why 


they  have  used  or  adapted  the  tools  they 
have  or  how  they  have  used  the 
journal's  participatory  principles  and 
philosophy  to  guide  their  assessment 
adventures. 

Before  introducing  each  author  to 
you,  I would  like  to  ask  for  your  indul- 
gence for  a moment  while  I sing  my 
swan  song — reflect  and  do  my  own 
critical  assessment  of  the  five  volumes 
and  where  I hope  and  think  the  journal 
should  go  in  the  future. 

As  stated  already  Adventures  in  Assess- 
ment is  a field-based  journal  dedicated  to 
writings  on  learner-centered  or  partici- 
patory approaches  to  assessment  and 
evaluation.  In  my  job  as  Literacy  Special- 
ist for  SABES  with  the  charge  to  provide 
technical  assistance  to  adult  literacy 
practitioners  and  pro-  grains,  it  was  my 
belief  that  the  field  needed  both  a frame- 
work for  investigating  participatory 
approaches  to  assessment  and  a forum 
for  expressing,  sharing,  and  document- 
ing ideas,  tools,  and  questions.  From  the 
beginning  these  two  strands  were  woven 
together  into  one  cloth-  "writing  about 
it".  Both  were  equally  important;  the  it- 
leamer-centered  approaches  to  assess- 
ment and  the  writing-  documenting 
inquiry. 

"It" 

Let  me  start  first  with  "it',  learner- 
centered  or  participatory  approaches  to 
assessment.  From  the  beginning  the 
journal  was  a way  to  reinforce  and 
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sustain  energy  my  workshops  on  assess- 
ment had  created-  a way  to  get  practitio- 
ners to  talk  and  listen  to  each  other 
across  the  state.  The  journal  was  a 
beginning  point  for  the  development  of 
the  toolkit  of  alternative  assessment 
approaches  and  tools  based  on  the  three 
phases  of  the  assessment  process.  And 
finally,  the  journal  was  an  attempt  to 
make  real  some  of  Susan  Lytle's  recom- 
mendations to  the  field  of  adult  educa- 
tion (Lytle,  1988).  Lytle  stated  that 
funders  and  legislators  use  standardized 
measurements  to  determine  program 
accountability  and.  sffsctivsr.sss  b6C£*us6 
they  lack  good  information  about  the 
qualitative  effects  of  programs  on 
learner's  lives.  She  advocated  two  basic 
strategies  to  remedy  this  situation:  the 
first  was  to  invite  a wider  participation 
into  the  conversation,  thus,  who  better 
than  the  literacy  practitioners  them- 
selves; the  second  was  to  conduct  pro- 
gram-based  practitioner  research  simul- 
taneously across  the  country  to 
strengthen  these  new  conceptual  frame- 
works and  to  exchange  and  critique 
innovative  practices. 

To  start  this  process  we  needed  a 
framework  with  clearly  articulated 
guiding  principles  so  that  the  word 
"alternative"  didn't  just  mean  anything 
other  than  standardized  tests.  As  Susan 
Lytle  (1989)  said,  "Constructing  new 
images  of  adults-  images  built  on  as- 
sumptions of  dignity  and  competence,  of 
literacy  as  reflective  and  self-critical 
practice,  and  of  learning  as  participatory 
— requires  that  we  rethink  or 
reconceptualize  not  only  our  notions  of 
what  counts  as  literacy  but  also  our 
methods  of  inquiry-  the  processes  we 
use  to  document  and  assess  learning". 

The  following  principles  are  a synthe- 


sis of  the  work  of  Lytle,  Fingeret,  and 
Auerbach,  my  mentors.  The  reader  will 
note  that  they  are  also  part  of  the  Massa- 
chusetts Participatory  Assessment 
Team's  mission  statement: 

Principles  of  Participatory  Assessment 

1.  It  must  be  program-based  and  learner- 
centered. 

2.  It  should  help  the  learners  achieve 
their  goals. 

3.  It  must  build  on  learner  strengths,  not 
deficits. 

4.  It  should  be  part  of  the  learning 
experience. 

5.  It  should  not  be  a single  procedure  but 
a variety  of  procedures. 

6.  It  should  provide  feedback  that  will 
lead  to  better  instruction. 

In  addition  to  these  principles,  to 
engage  in  truly  alternative  assessment, 
we  need  to  include  learners  as  active 
participants  at  the  center  of  the  process 
of  measurements  as  "co-investigators  in 
determining  their  own  literacy  practices, 
strengths  and  strategies"  (Lytle,  1988). 

As  I stated  in  Volume  II  in  talking  about 
portfolio  assessment,  "If  all  we  do  is 
substitute  new  multiple  measures  for  old 
standardized  measurements  and  moni- 
tor student  progress  for  diagnostic 
purposes  in  terms  of  identifying 
strengths  and  weaknesses  in  language 
and  content  areas,  we  will  not  have 
created  a new  paradigm." 

Paradigm  shifts  are  not  easy.  To 
support  this  shift  we  looked  at  the 
assessment  process  in  three  phases:  1.) 
Start  Up  or  Intake  Activities;  2.)  Along 
the  Way  or  On-going  Activities;  and  3.) 
Looking  Back  or  End  of  Cycle  Activities 
(Auerbach,  1992). 
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HISTORY  OF  THE  JOURNAL 

The  journal  began  in  the  Fall  of  1991. 
The  intention  from  the  beginning  was  to 
publish  three  journals  devoted  to  the 
three  phases  of  the  assessment  process. 
These  journals  were  intended  as  guides, 
resources  by  and  for  practitioners  to 
select  and  adapt  tools  for  their  own 
contexts. 

By  volume  II,  in  addition  to  introduc- 
ing the  authors,  I asked  the  reader  to 
think  about  the  different  "lenses" 
through  which  to  view  and  analyze  the 
writings.  I posed  questions  for  the  reader 
to  consider  like  "compare  the  evolution 
of  the  forms  and  tools  Paul  Trunnell 
developed  for  his  ABE  learners  to  the 
way  Kathy  Brucker  developed  her  tools. 
How  are  the  processes  similar?  How  do 
they  compare  to  the  kind  of  anecdotal 
reporting  Janet  Isserlis  does  daily  in  her 
classroom?  If  progress  is  achieved,  for 
whom  is  it  achieved  and  by  whom?"  In 
retrospect,  I see  in  these  words  the 
beginning  of  a shift  in  tone,  a caution  for 
people  not  to  just  adapt  someone  else's 
tool  but  to  remember  that  first  came  the 
toolmaker-  the  context,  then  the  tool. 

In  Volume  m.  Looking  Back,  I added 
three  new  features:  What  Counts?  (math 
assessment).  Voices  From  the  Field 
(interviews,  dialogues,  or  writings  from 
practitioners  who  are  not  classroom 
teachers,  and  Letters. 

I don't  recall  precisely  when  or  who 
made  the  decision  that  the  journal 
should  continue  beyond  the  original 
three.  This  feet  leads  me  to  believe  the 
field  had  started  to  count  on  the  journal 
as  a sure  thing  so  then  we  institutional- 
ized it  and  said  we  could  publish  two  a 
year-  one  in  the  fall  and  one  in  the 
spring.  However,  between  Volumes  HI 


and  IV  the  journal  lost  its  production 
editor.  This  forced  SABES  to  reconsider 
what  kind  of  editorial  support  we 
needed.  After  much  thought,  we  decided 
to  hire  an  editor,  Rick,  with  lots  of 
editing,  layout  and  graphic  experience, 
but  who  was  not  an  ABE  practitioner. 

His  task  was  to  redesign  the  journal  so 
its  format  was  more  consistent  with  and 
attractive  and  to  do  final  edits  with  the 
authors.  These  decisions  were  made  in 
part  because  the  journal  was  gaining 
popularity  outside  the  state  and  we  were 
moving  towards  selling  it.  One  of  the 
design  decisions  was  to  do  away  with 
the  appendices,  which  included  all  the 
forms  and  charts  practitioners  used. 
Though  at  first  this  seemed  like  a design 
decision,  it  really  became  a shift  in  what 
the  journal  was  supposed  to  do  for  the 
field.  By  integrating  the  charts,  forms, 
and  graphs  back  into  the  author's  piece 
we  were  hopefully  sending  the  message 
that  you  can't  just  go  out  and  take 
someone  else's  tool;  you  have  to  make  it 
your  own.  In  addition  to  these  graphic 
changes,  we  included  two  articles  from 
practitioners  from  out  of  state  and  a 
publication  review. 

WRITING 

"...Loren  asked  me,  reminded  me,  encour- 
aged me,  helped  and  revised  xvith  me." 

Don  Robishaw 

"..Jt  formalized  a process  that  had  been 
developing  with  much  discussion,  but  little 
or  no  documentation...  In  some  cases  loriting 
about  the  process  we  were  using  with  a 
particular  form  codified  something  that 
didn't  necessarily  make  much  sense  in 
practice.  In  others,  the  opposite  took  place- 
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something  that  didn't  make  sense  became 
obvious  in  the  process  ofzvriting  about  it.  I 
was  able  to  see  it,  or  dump  it  altogether." 

Janet  Kelly 

"We  started  zvith  an  initial  chat,  followed  by 
a sharing  of  the  forms  and  feedback  and  then 
a more  extensive  chat — "the  diner  diabgue" 
where  Loren  reviewed  my  work  and  sent 
back  a framework  for  preparing  and  pairing 
it  into  an  article." 

Paul  Trunnell 

In  addition  to  providing  a framework 
and  a forum  for  alternative  assessment,  I 
saw  the  creation  of  a field-based  journal 
as  a golden  opportunity  to  put  into 
practice  a process  approach  to  writing,  a 
way  for  practitioners  to  experience  first 
hand  the  power  of  having  their  writing 
responded  to  with  non-evaluative 
feedback.  Many  of  the  authors  in  Lindy 
Whiton's  research  survey  in  this  issue 
wrote  about  this  experience. 

In  general  I am  not  surprised  by  what 
these  authors  had  to  say  about  the 
journal  and  how  it  helped  them  reflect 
upon  their  practice.  I am  struck,  how- 
ever, by  a few  prevailing  comments  and 
sentiments.  The  first  is  embarassment 
over  how  important  my  asking,  prod- 
ding, pleading  was  to  get  people  to 
write.  Comments  like  "She  asked  me" 
and  "She  asked  me  to  submit"  make  me 
feel  both  a little  uncomfortable  with  my 
persuasive  abilities  and  at  the  same  time 
fissured  me  that  without  that  initial 
push,  many  wouldn't  have  done  it.  The 
second  is  the  power  of  what  happens 
when  you  write  something  down.  All 
the  authors  commented  on  the  self 
knowledge  they  gained  by  the  act  of 


writing  itself.  I had  underestimated  the 
power  of  writing  to  reflect  back  to  us 
what  we  really  think  and  believe.  And 
third,  I was  surprised  to  hear,  repeat- 
edly, the  desire  many  authors  expressed 
to  connect  with  others  so  they  could  get 
feedback  on  their  own  practice — "I  also 
thought  other  practitioners  might  think 
of  improvements,  and  I learn  from 
them" 

I am  also  struck  by  some  authors' 
clarity  about  the  need  to  get  this  infor- 
mation out  "I  want  presentation  portfo- 
lios to  be  used  everywhere.  I think  they 
are  marvelous!"  and  "I  wanted  to  share 
my  belief  that  self-assessment  is  not 
beyond  beginning  ESL  students;  that  it 
reduces  me  and  elevates  the  student  to 
co-creators  of  the  learning  environments. 
I wanted  to  support  the  cause  of  alterna- 
tive assessment  and  remind  people  that 
there  are  other  options  to  the  TABE." 

All  these  comments  reinforce  that 
writing  for  the  journal  has  been  an 
effective  means  for  doing  inquiry-based 
staff  development  as  well  as  developing 
the  knowledge  base  of  what  participa- 
tory assessment  looks  like  in  practice. 

So  now  that  you  have  listened  to  my 
own  reflections  and  critical  commentary, 
I would  like  to  invite  you  to  read  with 
pleasure  what  some  of  your  colleagues 
across  this  country  who  have  attended 
the  dream  conference  have  said  about 
the  journal 

We  begin  with  "One  Step  of  Inquiry: 
Documenting  the  V oices"  by  Lindy 
Whiton.  Whiton  conducted  a research 
project  last  year  to  find  out  if  the  process 
of  writing  for  the  journal  and  the  prod- 
uct itself  was  useful  staff  development 
and  whether  it  increased  the  field's 
knowledge  base.  Her  documentation  on 
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how  she  conducted  her  inquiry  and 
what  she  learned  from  both  the  reader 
and  writer  surveys  confirmed  many  of 
her  hypotheses. 

The  voices  of  teachers  in  Massachu- 
setts have  been  heard  in  Maine.  In 
"Hello,  Massachusetts",  Brawders 
outlines  what  the  vision  and  goals  are 
for  the  new  nationally-funded  Horizon 
Project.  According  to  Brawders,  Maine 
has  chosen  portfolio  assessment  to 
replace  standardized  tests.  Maine  is 
doing  what  they  call  "contextualized 
portfolio  assessment".  By  contextual 
they  mean  "each  adult  learner,  each 
teacher,  each  administrator  of  an  adult 
education  program  will  develop  a 
progress  portfolio  of  their  work  for  the 
year."  The  programs  will  then  measure 
their  qualitative  and  quantitative 
progress  in  relation  to  this  baseline  over 
a three-year  period.  What's  important 
and  exciting  here  is  that  programs  are 
not  measuring  against  some  specified 
norm  or  against  another  but  against 
themselves.  As  Brawders  says,  "What 
counts  is  the  articulation  of  action  steps 
that  can  be  taken  at  the  end  of  each  year 
for  the  purpose  of  program  improve- 
ment." 

Program  improvement  is  not  the  only 
goal  for  portfolio  assessment  as  Richard 
Goldberg  tells  us  in  "Portfolios  as  Alter- 
native Assessments  in  a Community- 
based  ESL  to  College  Transition  Pro- 
gram." Influenced  by  David  Rosen's 
article  "The  Progress  Portfolio"  in  AiA, 
Volume  2,  and  by  his  own  experience  in 
broadcast  journalism  with  students 
assembling  portfolios  of  their  work, 
Goldberg  decided  to  implement  them  in 
the  U.S.  Department  of  Education's 
Massachusetts  English  Literacy  Demon- 


stration project  (MELD).  The  MELD 
program  in  Chinatown  consists  of  three 
steps:  an  ESL  class,  an  Adult  Basic 
education  class,  and  then  enrollment  at 
Bunker  Hill  Community  College. 
Goldberg  teaches  the  ABE  class  and 
began  using  portfolios  to  document 
writing  progress.  The  turning  point  in 
the  MELD  program's  use  of  portfolio 
assessments  came  when  Bunker  Hill 
Community  College  agreed  to  use 
student  portfolios  as  an  alternative  to  its 
standardized  Comprehensive  English 
Language  Test  (CELT). 

Like  Goldberg,  Byron  Barahona  also 
got  ideas  from  the  journal  on  how  to 
adapt  and  create  tools  for  his  volunteer 
program  in  northern  California.  He  adds 
that  all  the  different  ideas  shown  in  the 
journals  have  influenced  his  vision  of  ‘ 
assessment  — from  diagnosing  prob- 
lems and  needs,  to  measuring  the  pro- 
cess of  learning  itself.  For  Barahona  both 
the  articles  and  the  tools  pose  important 
questions:  " First  and  foremost,  they 
make  us  think  and  rethink  the  need  and 
the  process  involved  in  evaluating 
progress.  Second,  they  make  us  look 
back  to  analyze  more  critically  what  has 
been  done.  Third,  they  help  us  reflect 
upon  what  can  be  changed,  adapted  or 
implemented."  His  article,  "Implement- 
ing Alternative  Assessment  Tools"  is  a 
wonderful  tribute  to  the  authors  he  has 
read  as  well  as  to  the  thoughtful  process 
for  real  adaptation 

In  "Images  of  Participatory  Assess- 
ment in  Adult  Education:  An  Analysis  of 
Adventures  in  Assessment",  Cathy  Luna 
echoes  Barahona  when  she  says  that  the 
reports  of  "experiments"  with  participa- 
tory assessment  practices  may  help 
practitioners,  learners  and  researchers 
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better  understand  the  nature  of  critical 
reflection  in  adult  education.  Luna's 
careful  and  thoughtful  analysis  of  the 
journal  focuses  attention  on  the  way  the 
authors  write  about  how  they  involve 
adult  learners  in  the  development,  use, 
and  revision  of  the  tools  and  processes 
used  to  assess  learner  progress  and 
program  evaluation.  She  asks  "In  what 
ways  can  practitioners  and  learners  talk 
and  work  together  to  build  frameworks 
for  critical  reflection  and  assessment?" 
Her  research  and  analysis  of  the  authors' 
writings  about  learner  involvement 


reve<u  Lliat  there  is  less  involvement  of 


learners  in  the  design  and  revision  of 
assessment  practices  than  in  their  imple- 
mentation. This  critical  feedback  is 
helpful  because  it  points  out  where  our 
strengths  are  and  gives  us  direction  for 
where  we  need  to  spend  more  energy. 

In  addition  to  these  critical  reflections 
are  writings  that  fall  under  our  estab- 
lished departments:  What  Counts? 

(math  assessment).  From  the  Field,  Book 
Review,  and  Letters.  The  reader  will 
note  that  there  are  a few  other  writings 
as  well.  Beginning  in  this  issue  and 
hopefully  to  continue  is  the  start  of  a 
new  column  called  "Learning  From 
Experience,"  designed  to  encourage 
practitioners  and  learners  to  write  about 
their  own  personal  experiences  being 
assessed  or  evaluated.  Two  other  writ- 
ings that  complete  this  Responding  to  the 
Dream  Conference  journal  are  a survey 
completed  by  a practitioner  from  West 
Virginia  and  the  mission  statement  from 
the  "Transformers",  the  Massachusetts 
Participatory  Assessment  Team. 

In  "Out  of  A Pickle:  Setting  the  Stage 
For  Math",  Martha  Merson  draws  upon 
her  experience  with  the  ABE  Math  Team 


teachers,  and  writes  about  how  to  create 
a "class  evaluation  that  doesn't  hurt  self- 
esteem, where  learners'  knowledge 
about  the  world  gets  counted  and  woven 
into  the  learning  at  hand,  where  work 
that  is  done  in  the  class  has  a purpose." 
She  reminds  us  that  we  need  to  educate 
our  students  to  the  alternatives  and  that 
we  can  do  this  by  laying  out  a strategy 
for  "familiarizing  students  with  a broad 
view  of  mathematics,  for  opening  the 
dialogue  about  what  topics  should  get 
covered  during  math  class,  for  using 
assessment  as  an  opportunity  to  build 
expectations  for  a new  or  continuing 
class." 

Marilyn  Gillespie  in  From  the  Field 
reminds  us  to  keep  the  dialogue  going 
and  to  try  to  find  better  ways  to  include 
learners  in  our  conversations.  She  also 
challenges  us,  those  who  write  for  the 
AiA,  to  reflect  upon  the  conditions  that 
affect  our  work  at  large  and  how  they 
could  be  changed. 

Anne  Marie  De  Martino's  letter  to 
Don  Robishaw  discusses  the  impact  his 
article  had  on  her  thinking  about  goal 
setting.  She  thanks  him  for  calling 
attention  to  the  need  to  place  goal  setting 
strategies  within  a context  of  the  greater 
philosophy  of  self-directed  learning  but 
disagrees  with  his  statement  that  "It  is 
very  important  that  the  facilitator  has 
had  similar  life  and  schooling  experi- 
ences as  the  learners  to  develop  solidar- 
ity with  them." 

In  "From  Minnow  to  Overachiever"  in 
Learning  From  Experience,  I write  about 
the  lasting  effects  of  being  evaluated  at 
different  points  in  my  youth  and  adult 
life. 

Don  Robishaw  reviews  a new  book  on 
portfolio  assessment  called  Portfolios  in 
the  Writing  Classroom.  Though  all  the 
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We  need  to  figure 
out  a way  to  bring 
the  learners  into 
our  conversation. 
We  need  to  hear 
their  voices.  We 
need  to  know  if 
our  new  concep- 
tual frameworks 
have  merit,  if  our 
new  approaches  to 
assessing  skills  are 
useful  and  if  our 
attempts  to  cap- 
ture their  gains 
meaningful. 


articles  are  written  by  and  for  teachers 
working  in  K-12,  he  recommends  the 
book  to  us  in  Adult  Education  who  are 
interested  in  portfolio  as  both  a tool  and 
a pedagogical  stance. 

And  finally.  Responding  to  The  Dream 
Conference  ends  with  the  Transformers 
mission  statement  and  strategies  for 
achieving  their  goals.  The  Transformers 
are  a group  of  ABE  and  ESL  practitio- 
ners who  are  "dedicated  to  concentrat- 
ing and  coordinating  efforts  towards 
broadening  the  understanding  of  adult 
educators,  learners,  administrators,  and 
funders  about  participatory  principles 
which  are  the  basis  for  meaningful 
assessment  and  social  change." 

Recommendations  for  the  Future 

As  part  of  my  final  reflection;  I would 
like  to  conclude  with  some  recommen- 
dations for  the  future  of  the  journal.  My 
first  recommendation  is  that  we  need  to 
figure  out  a way  to  bring  the  learners 
into  our  conversation.  We  need  to  hear 
their  voices.  We  need  to  know  if  our  new 
conceptual  frameworks  have  merit,  if 
our  new  approaches  to  assessing  skills 
are  useful  and  if  our  attempts  to  capture 
their  gains  meaningful.  "We  need  to  risk 
finding  out  not  just  how  our  learners 
experience  learning  but  how  we  help  or 
fail  them  in  that  process."  Second,  I 
would  also  like  to  figure  out  a way  to 
document  what  people  do  as  a group  or 
community.  I think  we  need  tools  and 
procedures  that  not  only  measure  indi- 
viduals but  communities  of  individuals 


working  towards  social  change.  Third,  I 
would  like  to  see  the  journal  take  a 
political  stand  and  support  native 
language  and  biliteracy.  I would  like  to 
see  articles  in  the  journal  on  how  to 
assess  native  language  literacy,  not  just 
for  diagnostic  purposes  or  as  way  to 
establish  a baseline  for  comparing 
English  language  acquisition,  but  as  a 
way  to  affirm  and  validate  the  belief  that 
our  learners  have  a right  to  be  bilingual 
and  bi-literate  and  that  we  believe 
English  is  a plus,  not  a substitution. 
Lastly,  I would  like  us  all  to  practice  a 
little  self-reflection  on  our  own  past 
experiences  at  being  assessed  or  evalu- 
ated. Were  these  positive  or  negative 
experiences?  Do  we  now  as  teachers 
impose  what  was  done  to  us  on  others, 
and  if  not,  what  informs  our  choice? 

In  general,  we  need  more  critical 
reflection.  As  Stephen  Brookfield  says, 
we  need  to  take  a reflective  stance  on 
our  practice  (both  past  and  present)  to 
be  clear  about  what  we  stand  for  or  are 
trying  to  achieve.  We  need  to  pose 
questions  not  just  seek  solutions  or 
create  tools.  We  need  to  become  skillful 
teachers  in  our  reflective  stance  on 
practice  and  our  dance  of  experimenta- 
tion and  risk-taking. 

Sources 

Auerbach,  Elsa.  Making  Meaning,  Making  Change . 
Delta  Systems,  Inc.,  McHenry  Illinois.  1992 

Brookfield,  Stephen.  Recognizing  Skillful  Teach- 
ing," unpublished  manuscript,  1993. 
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The  objective  of  this  paper  is  to 
describe  the  effect  documenting 
inquiry  has  on  practitioners  who 
write  for  Adventures  in  Assessment.  This 
is  a journal  dedicated  to  writings  on 
alternative,  learner-centered  assessment 
and  published  by  Massachusetts'  System 
for  Adult  Basic  Education  Support 
(SABES). 

The  purpose  of  this  research  came 
from  wanting  to  know  if  the  process  of 
writing  for  the  journal  was  useful  staff 
development.  It  has  developed  since 
then  to  also  ask  if  the  product  itself 
provides  opportunities  for  staff  develop- 
ment, and  whether  it  increases  the  field's 
knowledge  base. 

Many  questions  come  into  play 
when  people  talk  about  documentation. 
It  is  my  belief  that  a place  that  allows 
practitioners  to  document  their  work 
also  encourages  them  to  continue  to  do 
research.  It  is  also  my  belief  that  the  act 
of  doing  so  is  part  of  the  process  of 
inquiry  and  not  just  the  product  of  the 
research,  that  the  act  of  documenting 
encourages  teachers  to  look  more  closely 
at  what  they  have  done  and  to  reflect  on 
their  process. 

HOW  THIS  INQUIRY  WAS 
CONDUCTED:  METHODOLOGY 
As  originally  published  in  the  Intro- 
duction to  Adventures  in  Assessment, 
Volume  I,  the  journal's  creators  hoped  it 
would  become  a "resource  by  and  for 
practitioners  from  which  to  select  and 


adapt  tools  for  their  own  contexts." 
(McGrail  1991).  The  development  of  this 
resource  depends  upon  programs-based 
practitioners'  research,  the  results  of 
which  will  help  develop  the  field  of 
adult  education  (Lytle  & Wolfe  1989).  It 
was  felt  that  giving  practitioners  a place 
to  publish  would  not  only  encourage 
teacher-research,  but  be  a place  where 
their  work  could  be  validated  and 
shared  with  their  colleagues.  According 
to  Lytle  & Wolfe  (1989)  and  Auerbach 
(1992),  many  researchers  believe  it  is 
imperative  that  practitioners  begin  to 
document  their  inquiry,  to  record  their 
objections  and  to  engage  in  dialogues 
with  one  another  informing  their  col- 
leagues and  adding  to  the  body  of 
qualitative  research  in  education. 

First,  it  was  assumed  that  writing 
for  this  journal  would  be  good  staff 
development  Good  staff  development 
is  defined  as  the  process  of  building, 
reinforcing,  and  maintaining  effective 
teaching  practices.  Ultimately,  staff 
development  acts  as  an  antidote  to 
burnout.  Second,  the  actual  process  of 
writing  would  be  important  to  develop- 
ment of  the  field  (Gillespie  1991).  I felt 
that  the  actual  steps  in  revision  that 
authors  would  take  were  important  to 
their  personal  development  and  that  the 
act  of  publishing  would  be  validating. 
Third,  the  journal  would  increase  the 
knowledge  and  understanding  of 
alternative  assessment;  and  fourth, 
this  is  important  to  practitioners.  The 
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journal  offers  a stage  for  the  voices  of 
teachers  to  be  heard.  It  is  a place  where 
other  teachers  can  listen  not  only  to  the 
"tale  of  the  tools,"  but  to  the  narrative, 
the  story  of  "Who,"  "What,"  "Why," 
and  "How  come?" 

In  the  past  several  months,  an  at- 
tempt has  been  made  to  collect  data  on 
the  effect  of  the  journal,  to  discover  what 
both  authors  and  readers  thought  about 
Adventures  in  Assessment  and  was  it 
doing  what  we  assumed  it  would  do? 
When  an  article  was  published,  did  it 
validate  a practitioner's  work?  Did  it 
actually  help  to  spark  inquiry  in  ether 
programs?  It  is  this  data  that  I would 
like  to  share  with  others. 

All  21  people  who  contributed 
articles  to  the  journal  were  sent  letters 
and  surveys.  Only  eleven  people  re- 
turned the  surveys.  The  other  ten  au- 
thors were  contacted,  but  their  input 
never  was  completed  (see  appendix). 

Five  hundred  copies  of  the  Readers' 
Survey  were  sent  out.  The  actual  survey 
form  was  developed  by  McGrail,  two 
staff  members  of  SABES'  Central  Re- 
source Center,  and  myself  (see  Appendix 
2).  There  was  no  attempt  to  concentrate 
on  specific  groups  (e.g.  the  SABES' 
coordinators  mailing  list  versus  regular 
journal  readers).  The  objective  of  the 
Readers'  Survey  was  to  discover 
whether  practitioners  were  using  their 
colleagues'  writings  to  further  their  staff 
development  or  as  encouragement  or 
inspiration  to  do  their  own  plunge  into 
alternative  assessment.  Only  twenty- 
three  of  the  five  hundred  questionnaires 
mailed  were  returned,  less  than  5%. 


ANALYSIS 
Readers'  Surveys 

It  should  be  noted  that  not  only  was 
the  mailing  list  inappropriate  for  this 
survey,  but  the  tum-around-time  on  the 
survey  was  much  too  short.  In  approxi- 
mately a one- week  period,  twenty-three 
responded.  Out  of  that  group,  sixteen 
stated  they  had  never  heard  of  the 
journal.  Several  conclusions  can  be 
drawn  from  this  data.  First,  respondents 
find  it  easier  to  simply  check  the  box  and 
mail  the  forms  rather  than  spend  time 
reflecting  and  answering  auestions. 

o ’ * • - KS  *. 

Therefore,  those  who  could  just  check 
"No"  on  the  "Have  you  ever  read  this 
journal?"  question  had  the  easiest  task 
and  were  most  apt  to  send  it  back. 
Second,  unfortunately,  practitioners 
were  inundated  by  surveys  last  foil  and 
this  was  just  one  of  many.  It  should  be 
noted  that  while  attending  the  annual 
state  conference,  practitioners  did  ap- 
proach me  and  apologize  for  their 
absent-mindedness  in  not  responding  to 
the  survey.  Third,  the  mailing  lists  were 
deemed  inappropriate.  The  decision  was 
made  to  ask  each  SABES  coordinator  for 
the  names  of  ten  practitioners  to  whom 
they  had  personally  handed  the  journal. 
Surveys  will  ultimately  be  sent  to  those 
people,  supported  by  telephone  inter- 
views. In  the  interim,  copies  of  the 
journal  were  sent  to  those  individuals 
who  had  responded  they  were  unfa- 
miliar with  the  Adventures  in  Assessment. 

The  seven  responses  that  were  most 
informative  for  this  research  emphasized 
using  the  tools  and  peoples'  reflections 
as  models  to  adapt  in  their  own  class- 
rooms, which  is  a major  objective  of  the 
journal 
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Authors'  Surveys 

The  author's  answers  were  consid- 
ered in  reference  to  the  assumptions 
listed  above.  I examined  whether  our 
assumptions  were  repeated  in  their 
answers.  I also  wanted  to  know  if  the 
journal  was  useful  in  ways  other  than 
initially  predicted.  What  was  most 
interesting  in  this  process  was  that, 
while  we  attempted  to  show  that  the 
journal  was  a product  of  teacher-re- 
search, which  served  a very  important 
role,  it  was  actually  just  a snapshot 
which  helped  us  to  look  at  the  process  of 
inquiry  at  a specific  moment.  Authors 
referred  to  the  journal  as  "a  useful  tool." 

Many  of  the  authors  did  speak  of 
their  process  with  Loren  McGrail,  editor 
of  Adventures  in  Assessment,  as  important 
elements  in  the  writing  of  the  article. 
Loren's  behavior  was  reflective  of  her 
theoretical  beliefs  of  writing  instruction 
for  all.  She  treats  teachers  and  learners 
the  same.  These  comments  made  it  quite 
apparent  to  me  that  Loren's  input  and 
her  inclusion  as  co-researcher  was 
necessary  to  the  project.  She  brought  to 
the  paper  an  entire  body  of  data  she  has 
compiled  from  her  perspective  as  the 
editor  and  from  her  close  association 
with  the  authors  (see  Introduction). 


CONCLUSION 

Adventures  in  Assessment  is  per- 
ceived by  contributors  and  readers 
alike  as  good  staff  development.  It 
provides  practitioners  with  a process 
that  builds,  reinforces  and  maintains 
effective  teaching  practices.  It  is  also  a 
successful  vehicle  for  helping  teachers  to 
write  and  declare  their  voice.  "The 
journal  helped  me  to  believe  in  me." 

"I  have  a voice  and  it,  too,  is  worth 


listening  to."  "It  reinforced  my  beliefs." 
These  comments  are  representative  of 
the  types  of  responses  I received.  For  ten 
of  the  eleven  respondents,  the  act  of 
writing  for  the  journal  was  a successful 
staff  development  activity.  It  should  also 
be  noted  that,  of  the  respondents  famil- 
iar with  Adventures  in  Assessment,  the 
majority  used  the  journal  regularly. 
Thus,  we  believe  the  journal  fits  the 
definition  of  "good  staff  development." 

The  actual  process  of  writing  is  an 
important  one.  Publishing  is  valuable  in 
validating  educators'  voices.Most  of  the 
respondents  in  both  categories  talked 
about  sharing.  They  spoke  about  the 
importance  of  sharing  in  the  field  and 
the  place  the  journal  played  in  that.  They 
spoke  about  it  being  a place  in  the 
process,  not  the  product  at  the  end  of  the 
process.  In  his  responses,  Paul  Trunnell 
suggested  we  allow  more  dialogue  to 
happen  in  the  journal  itself.  "I  would 
like  to  encourage  the  journal  and  the 
field  towards  dialogue-based  on  dia- 
logue-intended work.  Particularly  now 
that  such  a range  of  tools  and  proce- 
dures has  been  discussed." 

According  to  Janet  Isserlis,  the 
journal  is  not  necessarily  a vehicle  for 
closure;  it  is  a way  to  lead  to  more 
questions. 

The  journal  does  create  a public 
space  for  teacher  writings.  This  public 
space  is  an  important  step  for  teachers  to 
now  take  (Cochran  and  Lytle  '93,  Gillespie 
' 90 ).  These  authors  never  spoke  directly 
about  the  publishing  as  being  validating, 
however.  They  were  much  more  aware 
of  the  sharing  with  others  as  being 
validating  in  and  of  itself  and  spoke  of 
the  act  of  writing  as  helping  to  "clarify", 
"review",  or  be  more  "succinct."  "I  was 
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inspired  by  the  work  done  by  Janet 
Kelley.  I was  reminded  of  the  value  of 
sharing  tools,  frustrations,  and  solutions 
and  thought  that  writing  an  article  was 
the  easiest  way  to  share  with  a larger 
group."  "I  also  thought  that  other  practi- 
tioners might  think  of  improvements, 
and  I would  learn  from  them."  The  effect 
would  "be  that  they  might  get  some 
ideas  about  reflecting  on  their  own  work 
in  ways  that  would  ultimately  be  helpful 
to  them  and  to  their  learners."  They 
spoke  of  the  journal  as  a useful  tool. 

The  journal  increases  ones  knowl- 
edge base.  This  is  evident  in  how  many 
adult  education  classes  are  now  using 
Adventures  in  Assessment  as  a classroom 
reading.  Continued  compilation  of  data 
from  readers  will  clarify  just  how  their 
knowledge  base  is  being  increased. 

In  regards  to  the  authors'  voices, 
some  saw  their  article  as  an  addition  to 
the  field;  others  did  not. 

There  is  a genuine  interest  in  the 
journal.  "Thanks  for  the  encouragement. 

. ."  was  a comment  frequently  noted. 
Other  comments  indicated  bona  fide 
enthusiasm  and  support  for  the  journal. 
The  following  comments  repeatedly 
surfaced  during  our  research:  "Let's  get 
more  people  involved."  "dialogue" 
"people  should  try. . ."  "I  appreciate  its 
presence." 

This  research  did  not  reveal  anything 
startling.  It  basically  confirmed  by 
hypothesis  about  the  process.  The  only 
surprise  for  me  was  that  practitioners 
did  not  feel  that  the  actual  article  was  an 
ending  point,  nor  a concrete  validation 
of  their  work.  Instead,  they  focused  on 
what  part  the  articles  played  in  the 
process  of  inquiry;  they  were  a part  of 
the  process,  one  point  along  the  trail. 


Although  that  finding  was  surprising  to 
me,  it  was  also  exciting  and  reaffirming. 
It  was  a stronger  indication  that  the 
journal  was  a stimulus  for  further 
thought  and  conversation. 

This  research  has  not  come  to  an  end; 
it  is  not  complete.  The  readers'  surveys 
are  a very  important  piece  of  the  work, 
and  I am  planning  to  continue  this 
research.  I believe  that  this  information 
will  not  only  support  the  publishing  of 
the  journal,  but  will  also  support  in- 
quiry-based staff  development  Not  all 
Ae  authors  who  responded  to  my 
questionnaire  were  involved  in  actual 
inquiry  projects,  however,  most  were 
involved  in  their  own  questioning 
process.  I hope  that  further  research  will 
not  only  support  the  claims  that  inquiry- 
based  staff  development  is  successful, 
but  it  will  also  indicate  more  clearly 
why.  Listening  to  the  readers'  responses 
should  shed  light  on  some  of  the  an- 
swers, or  at  least  help  us  to  form  new 
questions.  The  authors  of  "The  Tale  of 
the  Tools,"  Volume  5,  were  not  inter- 
viewed. Their  responses  will  be  enlight- 
ening because  many  of  those  articles 
came  straight  from  either  the  assessment 
or  math  inquiry  projects.  Their  perspec- 
tives may  be  slightly  different.  I hope  to 
keep  the  research  on-going  and  to 
provide  an  update  later  on. 

It  is  still  important  to  look  at  the 
changes  in  classroom  practices,  changes 
in  methodology,  and  learners'  success  in 
which  the  journal  may  be  inspirational. 
Possibly  more  important,  however,  is 
that  it  continues  to  provide  a place 
where  linkages  can  be  made,  where 
teachers  and  eventually  learners  can 
write  about  practices  in  assessment  and 
curricula  that  model  the  most  effective 
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different  roles  at 
different  times  for 
different  teachers. 
But  most 
importantly,  it 
provides  great 
linkages  and 
begins  to  end  the 
massive  problem 
of  isolation  across 
all  members  of  the 
adult  education 
community. 


programs.  Adventures  in  Assessment 
models  writing  those  practices  in  its 
various  steps  towards  publication.  The 
collaborative  efforts  of  the  editor,  assis- 
tant editor,  and  author  need  to  continue 
to  model  the  type  of  writing  process 
work  that  we  subscribe  to.  This,  in  turn, 
models  our  understanding  of  good 
assessment  practices  as  well.  Thus,  for 
me,  the  data  echoes  what  "these  voices" 
have  said,  that  Adventures  in  Assess- 
ment is  good  staff  development. 

POSTSCRIPT 

Now  that  I have  completed  this  part 
of  the  research,  taught  a mini-course  on 
alternative  assessment,  and  used  the 
journal  as  text,  I would  like  to  state  that 
staff  development  is  just  that  — Devel- 
opment. People  are  at  different  places 
with  different  needs.  There  were  nine 
regular  participants  in  my  class.  At  the 
end,  only  three  were  comfortable 
enough  to  take  a stab  at  writing  for  the 
sixth  edition.  I thought  in  the  beginning 
there  would  be  more.  However,  all 
group  members  had  read  all  five  issues 
and  found  pieces  to  respond  to,  whether 
verbally  or  in  their  process  of  developing 
assessment  for  their  classrooms. 

Another  example  of  the  outcome  of 
inquiry-based  staff  development  is  the 
"Mentor  Project"  that  the  Component  #3 
practitioners  have  now  developed  and 
begun.  They  are  finding  different  pro- 
grams are  at  different  points  of  the 
process.  They  need  to  fit  what  they  are 


doing  to  the  individual  program  needs. 
Adventures  in  Assessment  will  be  appro- 
priate to  use  for  many  points  of  this 
process.  The  journal  will  always  play 
different  roles  at  different  times  for 
different  teachers.  But  most  importantly, 
it  provides  great  linkages  and  begins  to 
end  the  massive  problem  of  isolation 
across  all  members  of  the  adult  educa- 
tion community. 
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This  survey  is  to  determine  the  iffect  that  Adventures  in  Assessment:  Learner  Centered 
Approaches  to  Assessment  and  Evaluation  has  had  on  the  field.  Does  the  journal  either 
change  or  validate  people’s  practices  and  is  it  a good  staff  development  tool?  We  define  staff 
development  as  the  process  of  reinforcing,  maintaining  and  building  effective  teaching 
practices.  Ultimately,  staff  development  acts  as  an  antidote  to  burnout. 


1 . Have  you  read  Adventures  in  Assessment? 


Yes  S No 


If  you  have,  which  volume(s)?  ^a.  Volume  1 (Yellow) 

ks  c.  Volume  3 (Purple) 


t'b.  Volume  2 (Light  Green) 
,/d.  Volume  4 (Dark  Green) 

j/'d-  volume.  £ Ctfj dci  ) 


3. 


Do  you  find  the  layout/design  of  Adventures  in  Assessment  accessible  or  easy  to 
understand?  Yes  v C No 

Comments: 

JCJ-I  Lee *rr*-*~~  srrvtx*- 

-A  Six.a_J2  ^ 

j yja  tfCcA  aALy  C*Z*  £*- 

/^Zo^s-  OAt-  «-»"«  Z . 7%-^  to  &>’ 

What  critena  <fo  you  use  when  deciding  which  articles  to  read?  /l  i 

jZ-~e£<Jhcs>*st 


Titles  or  articles  / 

Author  

Interests  j«< 

Other  (Specify)  


Comments: 


Do  you  read  the  introductions? 

Do  you  read  Voices  from  the  Field? 
Do  you  read  Gening  Started? 

Do  you  read  Ongoing? 

Do  you  read  What  Counts? 


J ste-a-A*  A 
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4.  Has  Adventures  in  Assessment  affected  your  thinking  or  beliefs?  Yes  t/  No 

Comments: 


1/ckJL^A 


CC4j1 


(2  cr/ 


'ri^e  .**3  £LC  Cj/  tl 


Ale.  de-c.*-  o.'tttJz  . 

6-a'  Q^tu.^4 --  •/—  #,l?6'  t't 'AjyVL44£-^  Ufcn  -7T-Z  A'  ^JjAr,  L-Le_ 

tSsrJl  Cu£ayJh&^dU  a*4jtx&sy<£- *d£  „ 

5-  H*ai  Adventures  in'A^osmem  erected  your  practices  tauch  a*  sparked  a new  tool  or 
thrown  out  aii  of  them)?  Yes  ✓ No 

jd<J  J^ctU- 

/VL*JI  M*?  c 

**  a.  LtfJUK  ' 

. jf*** 

6.  Have  you  changed  the  your  assessment  practices  in  your  classroom  and/or  program? 

<fyi  AJZA£MXl/y  &s7^  ^ 

" r/  \/  ^ /o/.a*i£  ~ZT  ° ^ CCs 

/Vil-ld- Ar7  / aA  AAcl. 

iv  „ v,  Z/iSWu  q 

7. . Do  you  have  ways  in  which  you  include  learners  in  your  assessment  practices? 


What  are  they?  „ , 

CTu^oiL.  *2" 

o ” CJb*<L4'<2^ 

lifV^UC  s?w'<d(U<j<u-i  - £?£U^aJ~  a-a.  «-> 

c/?  <64  -d £AML  Ab.  ^ ^ 

/ fhLcjbic^ y /bl  l,'C4J!  Pkv! /a, ’y^l' C*7», 


J AZl'tsLe  gggff  4MC<x^  farfJa,miyi~'C*?. 
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SAMPLE  SURVEY.  IF  YOU  WISH  TO  FILL  ONE  OUT,  SEE  PACES  75  AND  76 

This  survey  is  to  determine  the  effect  that  Adventures  in  Assessment:  Learner  Centered  Approaches  to  Assessment 
and  Evaluation  has  had  on  the  field.  Does  the  journal  either  change  or  validate  people's  practices  and  is  it  a good 
staff  development  tool?  We  define  staff  development  as  the  process  of  reinforcing,  maintaining  and  building 
effective  teaching  practices.  Ultimately,  staff  development  acts  as  an  antidote  to  burnout. 

1 . Have  you  read  Adventures  in  Assessment ? □ Yes  □ No 

If  you  have,  which  volume(s)?  □Volume  1 (Yellow)  □Volume  2 (Light  Green) 

□ Volume  3 (Purple)  □ Volume  4 (Dark  Green) 

2.  Do  you  find  the  layout/design  of  Adventures  in  Assessment  accessible  or  easy  to  understand?Yes  No 
Comments: 


3.  What  criteria  do  you  use  when  deciding  which  articles  to  read? 

□ Titles  or  articles  Do  you  read  the  introductions?  □ Yes  □ No 

□ Author  Do  you  read  Voices  from  the  Field?  □ Yes  □ No 

□ Interests  Do  you  read  Getting  Started?  □ Yes  □ No 

□ Other  (Specify)  Do  you  read  Ongoing?  □ Yes  □ No 

Do  you  read  What  Counts?  □ Yes  □ No 

Comments: 


4.  Has  Adventures  in  Assessment  affected  your  thinking  or  beliefs?  □ Yes  □ No 
Comments: 


5.  Has  Adventures  in  Assessment  affected  your  practices  (such  as  sparked  a new  tool  orthrown  out  all  of 
them)?  GYes  □ No 
Comments: 


6.  Have  you  changed  the  your  assessment  practices  in  your  classroom  and/or  program?  □ Yes  □ No 

7.  Do  you  have  ways  in  which  you  include  learners  in  your  assessment  practices?  What  are  they? 


8a.  Have  you  ever  used  any  ideas  or  tools  from  an  article?  □ Yes  □ No  Which  one? 
8b.  Please  comment  on  how  you  used  it.  Did  you  change  it?  Did  you  adapt  it? 


8c.  May  we  publish  these  comments  in  the  "Letter  to  the  Editor"  section  of  the  journal? 

□ Yes  ONo 

9.  Would  you  be  interested  in  writing  for  the  journal?  Yes  No 

If  yes,  please  fill  in  the  information  below  or  call  Loren  McGrail  at  World  Education,  (617)  482-9485. 

Address: 

Phone: 


Name: 


x^~\  XJ^hX 


Responding 

to  the  Dream  2 I 

Conference 


Portfolio  in  Maine 

Hello,  Massachusetts 


The  New  England  Literacy  Resource  Center  was  established  to  strengthen  adult  literacy  services  in  New 
England  by  promoting  and  facilitating  collaborations  and  sharing  among  adult  literacy  practitioners,  resource 
centers  and  policymakers  in  our  region.  NELRC  trill  be  doing  its  job  if  we  can  turn  the  information  flowing 
across  our  borders  form  a trickle  to  a stream  with  many  tributaries.  Alternative  assessment  is  one  area  of  great 
interest  across  New  England.  Clearly,  we  have  a lot  to  learn  from  each  other. 

I had  the  privilege  to  participate  in  the  kick-off  training  for  Maine's  Horizon  Project  in  which  portfolio 
assessment  is  central.  The  level  of  energy  and  enthusiasm  I witnessed  is  a good  match  to  the  ambitious  scope  of 
the  project.  Its  reporting  and  evaluation  dimensions,  in  particular,  tackle  some  of  the  greatest  utumsioered 
questions  about  alternative  assessment.  I am  eager  to  leant  from  Maine's  experience,  and  I hope  that  we  trill 
all  hear  updates  on  this  project. 

I hope  this  article  serves  as  inspiration  for  practitioners  across  New  England  to  explore  alternative 
assessment  and  to  contribute  to  the  growing  body  of  information  and  experience  on  the  subject.  May  this 
article  mark  the  beginning  of  sustained  collaboration  on  alternative  assessment  among  all  New  England 

states. 

— Silja  Kallenbach,  NELRC  Coordinator 


The  adult  education  network  in 

the  state  of  Maine  wants  to  thank 
all  the  teachers  for  the  excellent 
work  that  you  produced  in  the  five 
volumes  of  Adventures  in  Assessment  that 
we  have  received.  Thank  you  for  being 
honest,  persistent,  committed,  creative, 
flamboyant,  grounded  and,  above  all, 
diverse. 

All  of  us  in  Maine  are  hoping  for  a 
summit  of  assessment  in  the  next  12 
months  and  all  journal  contributors  will 
be  sent  an  invitation.  I would  like  to 
thank  Loren  McGrail  for  her  ability  to 
encourage  busy  practitioners  to  share 
their  work  for  the  greater  good  of  all 
adult  learners.  The  five  volumes  may 
become  a bestseller  in  Maine.  I am  using 
them  as  the  only  text  in  an  evaluation 
course  I am  teaching  through  the  Uni- 
versity of  Maine. 


"Escaping  flatland  is  an  essential 
task  of  envisioning  information  — for  all 
the  interesting  worlds  (physical,  biologi- 
cal, imaginary,  human)  that  we  seek  to 
understand  ar  inevitably  and  happily 
multivariate  in  nature.  Not  flatlands." 

— from  Envisioning  Information  by 
Edward  Tufte 

The  excitement  that  we  want  to  share 
with  you  is  that  Maine  through  its 
Quality  Indicators  has  chosen  portfolio 
assessment  to  replace  our  standardized 
tests,  peer  evaluation  as  our  program 
review,  teacher-based  research  to  affect 
public  policy,  and  a new  reporting  mecha- 
nism that  blends  qualitative  and 
quantitative  data  into  a usable  commod- 
ity for  program  improvement. 

Maine  is  doing  what  we  call  contex- 
tualized portfolio  assessment.  Each  adult 
learner,  each  teacher,  each  administrator 
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for  the  purpose 
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improvement . 


of  an  adult  education  program  will 
develop  a progress  portfolio  of  their 
work  for  the  year.  This  portfolio  will  not 
be  scored,  graded,  ranked,  or  put  in 
competition  to  beat  out  some  other 
program  for  funding.  The  purpose  of  the 
tiered  portfolios  that  build  on  each  other 
is  program  improvement. 

Each  program  in  Maine  is  develop- 
ing a baseline  of  who  they  are,  what  they 
offer,  what  they  have  for  funding,  how 
many  hours  teachers  work,  what  ser- 
vices are/are  not  in  the  community  to 
support  next  steps,  what  barriers  might 
be  insurmountable  for  the  adult  learners 
given  rural  issues  of  transportation,  etc. 

The  baseline  is  the  first  item  in 
everyone's  portfolio;  the  programs  will 
measure  qualitative  and  quantitative 
progress  in  relation  to  this  baseline  over 
a three-year  period.  The  program  mea- 
sures against  itself,  not  in  relation  to  a 
program  three  hours  away  with  four 
times  the  funding  and  a different  popu- 
lation! 

What  counts  is  the  articulation  of 
action  steps  that  can  be  taken  at  the  end 
of  each  year  for  the  purpose  of  program 
improvement.  The  steps  and  follow 
through  become  the  basis  of  the  self- 
evaluation  of  a peer  evaluation  process 
that  may  occur  every  three  or  four  years. 

In  their  roles,  teachers  see  them- 
selves change  through  their  portfolio 
depending  on  the  people  with  whom 
they  are  working.  Adult  learners  under- 
stand their  importance  to  improving  the 
program  for  the  next  year. 

Adult  learner  portfolios  are  only  one 
measure  of  a programs  goals  being 
reached.  The  yearly  action  plans  of  each 
program  double  as  their  end  of  year 
report,  and  gives  the  Staff  Development 


Team  specific  instructions  for  prioritiz- 
ing program  needs. 

Through  our  Horizon  Grant  with  the 
National  Institute  for  Literacy,  Maine 
has  seven  regional  trainers  working  on 
site  with  teachers,  adult  learners,  and 
administrators  to  create  this  integrated 
system  of  assessment,  evaluation,  re- 
search, and  reporting.  Reporting  will  no 
longer  be  a mystery  that  only  adminis- 
trators understand.  Adult  learners  and 
teachers  will  know  why  we  need  certain 
things  and  will  have  the  language  to 
question  whether  we  do  need  certain 
things! 

This  contextualized  portfolio  process 
gets  rid  of  the  secret  language  of  stan- 
dardized testing  results,  the  secret 
language  of  funding,  and  the  secret 
language  of  day  school  This  is  education 
among  peers!  Portfolio  allows  respect 
and  dignity  to  prevail! 

We  are  using  the  portfolio  process  as 
a real  curriculum  for  developing  critical 
thinking  skills  and  also  as  a vehicle  to 
demonstrate  transferable  skills. 

Some  programs  in  Maine  are  using 
video  portfolio,  portfolio  on  disc,  audio 
portions,  or  a combination  of  various 
media.  Jokes  are  circulating  that  we 
want  to  eventually  send  our  state  report 
to  the  federal  government  as  a hologram 
showing  the  quantitative  and  then  the 
qualitative.  I think  pop-ups  would  also 
get  some  attention. 

The  real  point  of  the  humor  is  that 
technology  is  giving  us  permission  to 
think  and  display  what  we  think  differ- 
ently. Yes,  there  are  CD-ROM  life-long 
learning  portfolios  that  are  not  just  text! 
Technology  will  change  the  way  we 
perceive  information.  To  believe  that  we 
will  not  find  a way  to  report  large 
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amounts  of  qualitative  data  from  portfo- 
lios when  we  have  a computer  with  500 
megabytes  of  memory  is  absurd.  We 
cannot  allow  a lack  of  imagination  to 
stop  us,  especially  if  it  means  a more 
truthful  document  upon  which  to  base 
the  development  of  public  policy  and 
funding  for  adult  learners. 

Join  us  in  this  great  adventure  and 
leave  flatland  behind.  Reports  are  meant 


to  be  read  and  anticipated,  evaluation  is 
meant  to  lead  towards  exciting  change, 
assessment  is  a vehicle  of  building  self- 
esteem and  documenting  progress,  and 
research  is  a way  to  check  standardiza- 
tion and  assumptions. 

If  only  I could  put  a pop-up  here, 
just  a small,  discreet  pop-up,  not  too 
flashy... 


EDUCATIONAL  AUTOBIOGRAPHIES 


for  Every  Level 


New 

Public 

Policy 


•individualized 

•measure  progress 
and  outcomes 

•determine  goals 
and  ongoing 
assessment 

•no  secrets 
cc:  learner  knows 
everything/has 
everything 

•process  taught 

•accomplishments 

demonstrated 

•ownership  by 
learner 


•changing 
roles  cap- 
tured 

•determine 
next  step  for 
next  year 

•evaluates 

materials 

content 

priorities 

•new  teacher 

training 

mechanism 


•this  is  about 
a teacher's 
role  with  the 
student 


•program 
goals  met 

•change 

determined 


•facilitate 

reporting 

•research 

questions 

•process  set 
up  for 
input  to 
state  plan 


•quantitative 

•qualitative 

analysis 

•interagency 

reporting 

•policy 

development 

•cross 

fertilization 


•determine 
need  for 
new 

inititatives 

•new  fund- 
ing levels 
supported 

•innovation 
and  research 
field  based 
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PORTFOLIO  ASSESSMENT  FOREVER  CHANGES: 

Content 

Methodology 

Evaluation 

The  Golden  Rules 

The  Agreed  Upon  Ethical  Framework  for  Maine  Adult  Educators 

1. 

The  adult  learner  controls,  owns,  and  designs  their  own  portfolio  with  the 
practitioner.  It  is  to  be  portable  and  transferable. 

2. 

The  learner  will  self-evaluate  their  own  portfolio  (progress  portfolio)  and  team  to 
seek  peer  evaluation  of  their  outcome  portfolio. 

3. 

Demonstration  of  progress  toward  the  learner's  long  and  short  term  goals  will 
take  place  within  the  portfolio. 

4. 

Portfolio  assessment  models  the  concept  of  process  thinking,  evaluation, 
sequential  thinking,  problem  solving,  data  gathering,  theorizing,  critiquing  and 
contrasting  ideas. 

5. 

The  completion  of  the  outcome  portfolio  is  a marker  in  the  adult  learner's 
process  of  life  long  learning. 

6. 

Portfolio  assessment  changes  the  role  of  the  teacher  to  that  of  a facilitator, 
researcher,  consultant,  peer  learner,  active  listener 

7. 

Teacher  enthusiasm  must  be  paramount  and  the  use  of  the  portfolio  should 
develop  as  a daily  educational  ritual  with  the  adult  learner. 

B. 

The  reporting  goal  of  the  portfolio  is  to  thread  both  quanitative  and  qualitative 
data  into  a more  realistic  picture  of  adult  learning  through  each  learner,  each 
teacher,  each  program,  each  state. 

Portfolio  assessment  is  a vehicle  for  capturing  the  developing  educational 
autobiography  of  each  adult  learner. 
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Portfolios  as  Alternative  Assessment  in  a 
Community-Based  ESL  Transition  Program 


This  article  concerns  an  idea  that 
is  still  evolving.  It's  an  idea  that 
was  implemented  in  January, 
1993,  but  was  hatched  several  years 
earlier  through  my  experiences  at  other 
workplaces.  It's  an  idea  that  exemplifies 
the  spirit  of  cooperation  between  two 
community-based  organizations  and  a 
community  college.  And.  it7?  ?_n_  id69.; 
similar  to  one  tried  in  other  places, 
which  shows  students  a lot  about  how 
they  can  see  their  progress  beyond 
traditional  standardized  tests. 

Background 

The  Massachusetts  English  Literacy 
Demonstration  (MELD)  was  established 
in  late  1992  through  a U.S.  Department 
of  Education  grant.  The  Massachusetts 
Department  of  Education  wrote  the 
proposal  for  three  partnerships  between 
community-based  organizations  and 
community  colleges. 

The  focus  here  is  on  one  of  those 
partnerships,  involving  the  Asian 
American  Civic  Association  and  Quincy 
School  Community  Council,  both  lo- 
cated in  Boston's  Chinatown  commu- 
nity, and  Bunker  Hill  Community 
College.  The  partnership  adopted  the 
acronym  ETP  (English  Transitional 
Program). 

The  MELD  program  in  Chinatown 
consists  of  three  steps:  an  ESL  class. 
Adult  Basic  Education  (my  class)  and 
then  enrollment  at  Bunker  Hill  in  pro- 
grams leading  to  either  a one-year 


certificate  or  a two-year  associate's 
degree.  Depending  on  their  English 
proficiency,  students  can  enter  the 
program  at  any  of  the  three  steps. 

At  all  levels  students  receive  counsel- 
ing throughout  the  transition  from 
Chinatown  to  Bunker  Hill,  which  lasts 
between  20  weeks  and  almost  one  year. 

Before  the  first  classes  began,  teachers 
and  program  administrators  agreed  that 
helping  students  build  an  on-going 
collection  of  their  work  was  worth  a try. 

Since  MELD  was  part  of  a national 
demonstration  project,  we  had  the 
freedom  to  experiment. 

Previous  Experience 

My  feelings  about  portfolio  assess- 
ments go  back  to  an  experience  in  my 
previous  career  as  a television  news 
producer  and  writer  in  Boston.  While 
teaching  broadcast  journalism  courses  at 
two  local  colleges,  it  was  easy  for  me  to 
have  students  assemble  portfolios. 

Among  other  things,  the  curriculum 
called  for  students  to  write  between  15 
and  25  news  stories  ranging  from  poli- 
tics to  economics  to  items  off  the  police 
blotter.  They  also  had  to  put  together  a 
five-minute  news  program  on  an  audio 
tape  cassette.  I instructed  students  to  use  Asian  American 

this  collection  of  their  work  as  a presen-  Qvic  Association 

tation  portfolio  similar  to  the  one  de- 
scribed below.  When  students  returned 
from  their  internships  at  radio  and  TV  < 
stations,  they  told  me  about  excellent 
feedback  from  their  employers.  Many 
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The  turning  point 
in  the  MELD 
program's  use  of 
portfolio  assess- 
ments came  when 
Bunker  Hill 
Community 
College  agreed  to 
use  student 
portfolios  as  an 
alternative  to  its 
standardized 
Comprehensive 
English  Language 
Test  (CELT) 


students  heard  similar  things  from  their 
bosses  such  as,  "These  examples  show 
me  you  know  enough  to  get  right  to 
work.  I don't  have  to  take  time  to  teach 
you." 

In  the  past  two  years,  I have  read 
some  of  the  recent  literature  about 
alternative  assessments,  beginning  with 
David  Rosen's  article.  The  Progress 
Portfolio  (see  Adventures  in  Assessment, 
VoL  2:  Ongoing,  May,  1992).  Especially 
helpful  were  the  resources  quoted  at  the 
end  of  David's  article.  At  that  time  the 
presentation  portfolio  model  and  its 
step-by-step  process  seemed  like  a 
natural  for  many  of  the  students  in  my 
class  at  a displaced  workers  transition 
center.  Unfortunately,  since  it  was  late  in 
the  cycle,  I didn't  do  anything  with  this 
idea. 

A publication  which  had  an  even 
greater  effect  was  It  Belongs  to  me  a Guide 
to  Portfolio  Assessment  in  Adult  Education 
Programs  (Fingeret,  1993)  (reviewed  by 
Steve  Reuys  in  Adventures  in  Assessment, 
VoL  5:  The  Tale  of  the  Tools,  October, 
1993).  Fingeret's  work,  which  crystal- 
lized many  of  my  beliefs  about  alterna- 
tive assessment,  is  a great  "how-to-do-it" 
from  people  who  have  done  it.  Of 
greatest  value  for  me  was  the  section  on 
moving  materials  from  writing  folders  to 
portfolios  and  how  students  assess  their 
collections  on  a regular  basis  and  make 
the  choices  as  to  what  constitutes 
"progress." 

Beginnings 

During  the  first  cycle  of  the  MELD 
class,  I was  fortunate  to  have  a group  of 
students  who  liked  to  write,  so  writing 
became  the  focus  of  their  portfolios.  The 
ABE  course  content  is  theme-based  and 


students  generate  most  of  the  themes, 
such  as  the  American  college  system, 
health  care,  workers'  rights,  the  legal 
system,  and  American  government, 
economics,  and  culture. 

Throughout  much  of  the  cycle, 
students  were  writing  a composition  a 
week,  which  gave  us  plenty  to  look  at 
once  we  began  the  process  of  moving 
material  from  writing  folders  to  their 
portfolios.  Toward  the  end  of  the  cycle, 
one  of  the  issues  that  surfaced  was  fear 
of  failure,  especially  in  a college  class- 
room the  students  next  step. 

"Certainly,  I worry  about  the  college 
class.  I ask  myself:  If  I go  to  Bunker  Hill,  can 
I understand  the  teacher's  lecture?  Can  I 
finish  the  homework?  Can  I qualify  for  the 
test?  Can  I Oh,  I don't  know." 

One  of  the  ways  we  dealt  with  these 
fears  was  to  have  students  conduct  a 
self-assessment  to  show  them  how  they 
improved  in  almost  23  weeks.  ( See  Fig.l, 
Progress  Checklist).  In  subsequent  cycles, 
the  self-assessment  was  done  at  the  mid- 
point and  again  at  the  end  of  the  course. 

There  were  other  things  in  students' 
folders.  ( See  Fig.  1-A,  From  Folders  to 
Portfolios).  One  woman  kept  a running 
list  of  activities  by  skill  area:  reading, 
writing,  listening,  and  speaking.  We  also 
had  a math  test  with  no  computation, 
only  brief  descriptions  in  English  of 
math  terminology  (the  students'  often- 
articulated  math  weakness),  such  as 
perimeter,  area,  volume,  radius,  diam- 
eter, and  circumference  Folders  also 
included  copies  of  Individual  Education 
Plans  (IEPs),  which  were  updated  with 
the  program  counselor  about  every  six 
weeks.  In  many  cases,  students'  educa- 
tional goals  became  more  sharply  fo- 
cused through  these  documents  (see  Fig. 
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2,  Individual  Educational  Plans). 

All  of  this  presented  a big  problem 
as  we  tried  to  wrap  things  up  in  the  last 
two  weeks  of  the  cycle.  There  was 
simply  too  much  material  in  the  folders 
and  too  little  time  for  students  to  reflect 
on  what  was  meaningful  as  an  indicator 
of  improved  English  skills.  In  the  end, 
each  student  decided  what  he  or  she 
would  put  into  the  portfolios.  Since  this 
was  the  first  time  we  were  using  portfo- 
lios, the  criteria  we  used  included  ques- 
tions such  as,  "What  kinds  of  things  in 
your  folder  would  give  a teacher  at 

T-Ttll  <5  /YAArl  AlAtlirO  nf  \7A1  IT* 

1/bUlACA  X mi  U j-  ■»  - wa  J v«AA 

ability  to  use  English?"  "What  things  in 
your  folder  show  that  you  used  English 
well?"  Among  their  entries  were  the 
kinds  of  compositions  which  they  might 
be  asked  to  do  in  college,  such  as  com- 
pare and  contrast  or  taking  a point  of 
view  and  defending  it  (one  example, 
"would  you  disobey  a law  which  you 
believed  was  unfair?").  Then  copies 
were  made.  Original  documents  were 
returned  to  each  student.  A completed 
application  form  for  admission  to  Bun- 
ker Hill  was  included  and  the  portfolios 
were  turned  over  to  the  college. 

Portfolios  at  Bunker  Hill 

The  turning  point  in  the  MELD 
program's  use  of  portfolio  assessments 
came  when  Bunker  Hill  Community 
College  agreed  to  use  student  portfolios 
as  an  alternative  to  its  standardized 
Comprehensive  English  Language  Test 
(CELT),  which  is  given  to  non-native 
English  speakers  who  apply  for  admis- 
sion. The  entire  process  of  convincing 
college  officials  involved  a discussion 
between  Alan  Shute,  MELD'S  program 
liaison  at  Bunker  Hill,  and  Ralph  Radell, 


chairman  of  the  school's  ESL  Depart-  ‘ 
ment.  Alan  is  a full-time  employee  who 
works  out  of  Bunker  Hill's  Center  for 
Self-Directed  Learning.  He  also  teaches 
an  ESL  course  at  the  college  and  be- 
comes the  advisor  to  all  MELD  students 
once  they  register  for  courses.  Long 
before  they  get  there,  he  introduces 
students  to  the  resources  available  at  the 
Learning  Center,  usually  in  the  second 
week  of  each  cycle.  His  presence  is  a key 
element  of  our  program  and  makes  the 
Chinatown  connection  to  the  school 
much  more  visible  and  credible.  Once 
students  enroll  at  the  college,-  they  can 
take  many  courses  in  the  Learning 
Center  as  well  as  in  a traditional  class- 
room. 

The  first  time  Radell  looked  at 
students'  portfolios,  he  was  not  given 
their  CELT  scores.  He  was  concerned 
about  portfolio  entries,  for  example,  not 
knowing  how  many  times  their  essays 
had  been  revised.  A writing  sample 
done  under  a 30-minute  deadline  on  the 
same  day  as  the  CELT  is  given  high 
priority  in  the  evaluation  of  borderline 
test  scores.  After  the  next  cycle,  more 
and  systematized  information  was 
provided.  This  time,  CELT  scores  were 
included,  along  with  results  of  each 
student's  work  in  the  Learning  Center 
over  the  past  six  months,  attendance, 
and  a student  profile  giving  some  of 
Alan's  personal  observations  for  a more 
complete  picture  of  the  student's  abilities 
and  motivation.  Students  are  actively 
involved  in  the  decision  on  where  they 
will  be  placed.  Before  they  register  for 
courses,  they  talk  with  Alan  about  the 
challenge  and  pace  of  the  courses  in 
which  their  portfolios  placed  them.  For 
the  most  part,  students  are  happy  with 
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their  placement  by  portfolio,  although  in 
a few  cases,  students  have  felt  more 
comfortable  starting  in  the  lower  level 
courses  where  their  standardized  test 
scores  placed  them  ( see  Fig.  3,  BHCC 
Results). 

Lessons  of  the  First  Year 

With  each  cycle,  the  process  of 
introducing,  implementing,  and  manag- 
ing portfolios  as  alternative  assessments 
gets  easier,  since  all  of  us  are  sold  on  the 
idea  that  portfolios  offer  a more  com- 
plete, in-depth  picture  of  students  than 
their  test  scores.  The  concept  is  still 
introduced  during  the  first  week  of 
classes  in  Chinatown.  Before  students  in 
the  ESL  class  move  up  to  the  ABE  class, 
they  present  a portfolio  of  their  work  to 
me.  Now  we  try  to  show  students  that  a 
portfolio  is  a way  for  them  to  measure 
their  progress  over  a period  of  time. 
There  is  also  an  incentive  for  students, 
since  we  remind  them  Bunker  Hill  gives 
their  portfolios  equal  weight  to  the 
standardized  college  placement  test. 
MELD  program  graduates  who  are  now 
students  at  the  college  come  back  to  talk 
with  current  students  and  often  mention 
how  they  felt  comfortable  with  the 
academic  ESL  level  placement  made  by 
their  portfolio. 

Subsequent  classes  do  essays  during 
the  first  week  of  the  cycle  on  defining 
goals,  which  become  the  first  entries  in 
their  portfolios  (see  Fig.4,  Defining  Goals). 
We  give  students  more  time  to  reflect  on 
what's  in  their  folders.  One  ABE  class 
took  almost  one  hour  to  answer  these 
questions  12  weeks  into  the  cycle: 

What  in  your  folder  tells  you  that  you 
have  made  progress? 

How  do  you  know? 


What  other  things  that  you  have  done 
(not  in  the  folder)  also  tell  you  that  your 
English  has  improved? 

What  else  would  you  like  to  add  to  your 
folder?  (See  Fig.  5,  Portfolio  Evaluation). 

This  was  a big  change  from  the 
previous  kinds  of  biweekly  class  evalua- 
tions I used,  short  answers  (I  liked ; I did 
not  like ) and  checklists,  which  were 
often  rushed  at  the  end  of  Friday  classes 
and  rarely  told  me  more  than  I already 
knew. 

We  also  allow  more  time  for  reflec- 
tion at  the  end  of  each  cycle,  when 
students  notice  some  of  the  biggest 
changes  and  1 see  the  biggest  rewards  of 
portfolio  assessments  ( see  Fig.  5:  How  Did 
I Improve?). 

The  Future 

As  we  near  the  end  of  the  second 
and  final  year  of  federal  funding  for  the 
MELD  program,  we  continue  to  refine 
the  process  of  portfolio  assessments. 
Now  the  portfolios  are  vised  not  only  as 
a presentation  portfolio  for  enrolling  at 
Bunker  Hill,  but  also  as  an  on-going 
document  of  students'  work  once  they 
get  there.  As  their  advisor,  Alan  tracks 
their  progress  and  adds  to  the  portfolio 
teachers'  mid-semester  evaluations  or 
recommendations  for  early  intervention 
if  a student  is  at  risk.  Also  included  are 
course  schedules,  transcripts  and  re- 
quirements for  each  student's  chosen 
certificate  or  degree  program.  The 
portfolio  is  then  used  when  Alan  advises 
students  during  registration  periods. 

After  just  one  semester  the  results 
are  encouraging.  Only  one  student  said 
she  struggled  with  her  placement  ac- 
cording to  the  portfolio,  but  she  passed 
the  course.  Another  student  who  placed 
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into  non-ESL  academic  courses  gave 
birth  to  her  first  child  just  before  final 
exams  and  will  have  to  make  up  incom- 
plete work.  All  the  MELD  students 
moved  up  the  following  semester,  a 
tribute  to  their  motivation  and  commit- 
ment. Nine  more  graduates  of  our 
classes  in  Chinatown  joined  them  at  the 


college  in  January,  1994.  There's  also 
evidence  that  portfolio  assessments  are 
taking  hold  in  other  parts  of  Bunker  HilL 
Some  teachers  in  the  college's  non-credit 
ESL  division  have  followed  MELD'S 
example  and  are  encouraging  their 
students  to  develop  portfolios  as  an 
alternative  placement  to  the  CELT. 
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Figure  1 

PROGRESS  CHECKLIST 


Hy  reading  comprehension  of  English  has  improved 

a great  deal 

a fair  amount 

not  very  much 

not  at  all 

When  I read  something  in  English  now,  I can 

(check  ail  of  the  things  you  can  do  better  than  oefore) 

understand  the  main  idea  of  what  I read. 

give  a short  oral  summary  of  what  I read. 

_ give  a short  written  summary  of  what  I read . 

use  my  background  knowledge  of  the  subject  to  help 

me  understand  what  I read. 

guess  the  meanings  of  some  new  words  from  how  they 

are  used  in  a sentence. 

Hy  writing  skills  have  improved 
_ a great  deal 

a fair  amount 

not  very  much 

not  at  all 

When  I write  an  essay  now,  I can 

(check  all  of  those  things  you  can  do  better  than  before; 

brainstorm  several  ideas  about  the  subject. 

organize  my  ideas  before  writing. 

follow  directions  and  write  on  a specific  subject. 

write  sentences  that  are  clear  and  easy  to  understand. 

use  correct  punctuation  and  capitalization. 

find  my  grammar  mistakes  and  correct  them. 

write  an  essay  under  a time  limit. 

Hy  speaking  skills  have  improved 

a great  deal 

a fair  amount 

not  very  much 

not  at  all 

When  I speak  English  now,  I can 

(check  all  of  the  things  you  can  do  better  than  oefore) 

speak  clearly  so  that  an  English  speaker  can 

understand  me. 

ask  an  English  speaker  questions  when  I do  not 

understand  him  or  her. 

use  English  more  in  the  classroom  to  speak  with 

teachers  and  classmates. 

_ give  the.  correct  pronunciation  of  most  English  words.  . 
know . 
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My  listening  and  note-taking  skills  have  improved 

a great  deal 

a fair  amount 

net  very  much 

not  at  all 

When  I listen  to  the  teacher  or  to  a tape  now,  I can 
(check  all  of  things  you  can  do  better  than  before) 

understand  the  main  idea  of  what  the  person  is  saying. 

understand  most  of  what  the  person  said  in  the  lecture 

or.  the  tape  . ( 

take  notes  using  my  own  abbreviations  or  ' shorthand. 

answer  questions  correctly  about  what  the  speaker  said. 
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From  Folders  to  Portfolios  in  the  First  MELD  Class 

* Writings 

* Self-assessment 

* Activities  by  skill  area 

* Math  test 

* Individual  Education  Flan  (IEF) 

* Completed  Bunker  Hill  admission  application 
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INDIVIDUAL  EDUCATiC 

>N  PLANS 

( ISP's) 

NAME:  S.X. 

DATE: 

Sept.  2 

GOAL:  Medical  Secretary 

WHEN:  1994 

STEPS 

ACHIEVED 

1.  Finished  ESL  class 

4/93 

2.  Move  up  to  ABE  class 

5/93 

3.  Will  go  to  Bunker  Hill  Community 
College 

9/93 

4.  will  register  for  BHCC 

9/7/93 

ACTIONS 

1*  Listen  to  the  radio  and  watch  the  TV  to  improve  my  listening. 

2.  Read  more  book  to  know  more  words. 

3.  Talk  more  to  improve  my  speaking. 

BARRIERS 

Listening  and  vocabulary  are  still  my  barriers. 
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NAME : H . L . 

GOAL:  After  I get  a high  school  certificate,  I will  apply  for 

college  to  study  electronics. 

WHEN: . ? 

cmroc  ACHIEVED' 


1. 

Enroll  ABE  class 

7/93 

2 . 

Go  to  EDP  (External  Diploma  Program) 

?/94 

at  AACA. 

3 . 

Apply  for  college  to  study 

?/9  5 

electronics  program. 

ACTIONS 

1.  Working  on  grammar,  writing,  essays  and  discussion. 

2. * Watch  TV  and  finish  the  homework. 

3.  Go  to  BHCC  to  use  computer. 

BARRIERS 

Time  problem  (restaurant  cook) 
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Figure  3 

Most  of  the  students  making  the  transition  to  Bunker  Hill 
Community  College  from  Chinatown  place  in  one  of  three 
levels  of  academic  ESL  courses,  one  being  the  lowest  and 
three  being  the  highest  and  last  level  of  ESL  before  students 
can  enroll  in  regular  academic  courses.  In  many  cases,  the 
alternative  portfolio  assessments  placed  students  one  level 
higher  than  they  were  placed  by  the  CELT  (Comprehensive 
English  Language  Test),  which  the  college  uses  as  its 
standardized  placement  test  for  non-native  English  speakers. 
These  assessments  were  done  in  August,  1993  for  students 
planning  to  enroll  for  the  September,  1993  semester.  Once 
they  enrolled,  all  but  three  took  courses  at  the  level  of 
their  portfolio  assessments. 

Placement  According 

Placement  According  to  to  Portfolio 

P.tn  riant  Co1,  logo  jg  c fonHayH i Test  Assessment. 


T.L. 

ESL  Level  1 

ESL  Level  2 

J.W. 

ESL  Level  2 

ESL  Level  3 

H.W. 

ESL  Level  1 

ESL  Level  2 

S.X. 

ESL  Level  3 
(This  student 
Levels  2 and 
Level  2). 

ESL  Level  2 

was  right  on  the  line  between 
3.  She  asked  to  be  placed  in 

Q.L . 

ESL  Level  3 

ESL  Level  3 

Q.K . 

ESL  Level  3 

ESL  Level  3 

M.C. 

ESL  Level  2 

ESL  Level  2 

F.L. 

ESL  Level  1 
(This  student 

ESL  Level  2 

asked  to  be  placed  in  Level  1), 

C.F. 

ESL  Level  1 

ESL  Level  2 

Y.T. 

ESL  Level  1 

ESL  Level  2 

K.C . 

ESL  Level  2 

ESL  Level  3 

P.C. 

ESL  Level  2 

ESL  Level  3 

M.H. 

ESL  Level  1 
(This  student 

ESL  Level  2 

asked  to  be  placed  in  Level  1). 

B.H. 

ESL  Level  2 

ESL  Level  2 
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Figure  4 

Excerpts  from  first  essays  on  defining  goals 
(written  during  first  week  of  cycle) 


"First  of  all  I should  learn  English  hard  and  finish  my 
homework  on  time.  Secondly,  I must  try  to  speak  more  English 
anytime  and  go  to  Bunker  Hill  Community  College’s  Learning 
Center  discussion  group  once  a week.  Finally,  I am 
determined  to  overcome  the  barriers  that  stopped  me  from 
learning.  I'll  watch  TV,  listen  to  the  radio  tape  and  read 
newspaper's  news  every  day.  If  I have  any  questions.  I'll 
ask  the  teacher  and  classmates.  I hope  after  finish  the 
class  my  English  will  move  into  a new  level.  I'll  be  a 
college  student  soon." 


"At  public  occasions  I was  unable  to  express  my  idea.  I 
felt  that  I was  almost  a person  with  a mouth  that  could  not 
speak  and  ears  that  could  not  hear.  Now  I face  a big 
change... The  teacher  encourages  us  with  a sentence  "A  journey 
of  a thousand  miles  must  begin  with  a single  step." 

Confidence  is  to  trust  myself.  Confidence  will  bring 
infinite  strength... I know  everybody  had  peaks  and  valleys  in 
the  life.  I seem  like  to  be  in  my  valleys  now.  I have  got 
to  have  patience  and  faith.  In  this  class  I learn  step  by 
step,  word  by  word.  I study  time  by  time.  Things  are  going 
to  turn  around  for  me...” 
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Figure  5 

When  you  look  at  what  is  in  your  portfolio,  how  can  you 
tell  your  English  has  improved? 

(written  during  week  17  of  a 23  week  cycle) 


"Since  I started  learning  at  this  class,  I feel  my 
English  has  improved.  Such  as  when  I watch  TV  and  listen*  to 
the  radio,  I can  understand  more  than  before.  And  also  I 
have  more  confidence  to  talk  with'other  people.  Such  as  when 
I go  outside  I can  talk  with  someone  and  on  the  telephone." 


"I  am  working  on  grammar,  writing,  essays,  reading, 
idiom  and  comprehension  in  ESL  Transitional  Program.  I 
visited  Bunker  Hill  Learning  Center  once  a week  discussion 
group  and  worked  on  the  computer... I have  improved  my  English 
a lot... I can  talk  English  with  American  people  at  work.  My 
reading  and  writing  skills  are  also  better  than  before." 


“The  time  passed  very  fast,  our  study  is  almost 
finished.  When  I reviewed  my  achievement,  I found  that  I 
made  great  progress  in  ray  English  studies. 

For  instance,  my  reading  skill  is  improving.  I read  the 
newspaper,  letters  and  books  more  often  than  before,  and  my 
writing  is  improving  too.  Although  I feel  writing  is  still 
difficult  for  me,  but  I still  keep  doing  it,  after  I finish 
each  essay  I feel  very  happy... 

I'm  very  sorry  that  the  class  end  very  soon,  and  we'll 
go  to  Bunker  Hill  College  to  continue  our  study,  it  will  be 
another  exciting  day!" 
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Assessment  in  California 

Implementing  Alternative  Assessment  Tools 


by 

Byron  Barahona 


Alameda  County  Library 
Adult  Literacy  Program, 
Fremont,  California 


T would  like  to  start  by  expressing 
my  appreciation  for  all  the  work 
.«>.  and  ideas  of  those  involved  in  the 
creation  of  this  journal,  especially  to 
teachers  who  contributed  articles  and 
alternative  assessment  tools.  I would  like 
to  offer  my  response  to  the  tools  pre- 
sented in  previous  journals  and  particu- 
larly how  they  have  been  adapted  to 
address  the  needs  of  the  learning  envi- 
ronment where  I teach.  Moreover,  I 
will  attempt  to  conceptualize  the  extent 
to  which  the  journals  have  affected  and 
influenced  my  vision  of  assessment. 

It  is  important  to  mention,  however, 
that  all  the  different  ideas  shown  in  the 
journals  made  me  evaluate  my  notion  of 
assessment  and  functioned  to  create  and 
implement  mechanisms  that  may  enable 
me  to  evaluate  learners'  progress  more 
efficiently  and  accurately.  The  range  of 
issues  oscillates  from  diagnosing  prob- 
lems and  needs  to  measuring  the  process 
of  learning.  Some  tools  were  not  particu- 
larly applicable,  yet  they  acted  as  elici- 
tors  to  analyzing  and  measuring  the 
effectiveness  of  the  different  assessment 
tools  presented. 

The  California  Program 

The  adult  literacy  program  I work  in 
is  located  in  Alameda  County,  northern 
California.  This  program  operates  with 
volunteers  who  commit  to  tutor  ABE  or 
ESL  students  for  two  to  four  hours  a 
week  for  a minimum  of  six  months. 
These  tutors  are  first  trained  by  our  staff 


for  18  hours.  Students  or  adult  learners 
who  come  to  this  program  are  expected 
to  be  able  to  focus  more  on  reading  and 
writing.  Likewise,  ESL  students  are 
expected  to  have  a minimum  level  of 
conversational  English  and  some  basic 
knowledge  of  English  structure. 

As  the  number  of  students  and 
tutors  grows,  there's  also  an  increasing 
need  to  find  more  adequate  means  to 
maintain  records  of  students'  progress. 
This  need  has  led  to  pursuing  new  and 
more  diverse  assessment  tools.  In  this 
search.  Adventures  in  Assessment  journals 
have  been  timely  for  pedagogical  con- 
cern and  key  to  the  success  of  reaching 
this  goal,  for  both  articles  and  assess- 
ment tools  found  in  the  journals  pose 
important  questions. 

First  and  foremost,  they  make  us  — 
practitioners  — think  and  rethink  the 
need  and  the  process  involved  in  evalu- 
ating progress.  Second,  they  make  us 
look  back  to  analyze  more  critically  what 
has  been  done. 

Third,  they  help  us  to  reflect  upon 
what  can  be  changed,  adapted  or  imple- 
mented. Finally,  while  there  is  a large 
degree  of  accountability  in  the  process, 
there  is  also  an  attempt  to  withdraw 
from  the  notion  that  only  standardized 
tests  can  accurately  provide  the  informa- 
tion needed.  This  balance  is  essential 
since  both  students  and  tutors  or  teach- 
ers want  and  need  to  know  what 
progress  has  been  made. 

The  following  are  some  of  the  tools 
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that  have  been  incorporated  into  the 
tutor  training.  The  selection  was  rather 
difficult  to  make  because  the  options 
were  many  and  of  important  value. 
However,  due  to  the  nature  of  our 
program  we  are  unable  to  expect  exten- 
sive and  more  comprehensive  assess- 
ment from  the  tutors.  Thus,  while  there 
was  a clear  attempt  to  limit  the  amount 
of  assessment  tools,  much  effort  was  put 
to  incorporate  the  most  appropriate 
tools.  Now  let's  look  at  some  of  the 
alternative  tools  chosen. 

The  progress  profile,  discussed  in 

LUUiic  roiiucx  o cuuuc  */*  1 %jt*C*l* 

Participants  Goals  and  issues'  (November 
1991),  is  a good  example  of  a tool  that 
actively  engages  the  learner.  It  has  the 
power  to  pose  questions  only  students 
may  be  able  to  answer.  It  calls  for  reflec- 
tion in  a manner  that  goes  beyond  the 
general  desire  of  simply  wanting  to 
improve  writing  and  readings  skills.  In  a 
Progress  Profile  the  learner  is  invited  to 
think  about  what  she  or  he  wants  to 
learn,  why,  and  its  feasibility.  Once  these 
three  steps  are  completed,  there  is  the 
opportunity  to  stop  and  reflect  on  its 
effectiveness  along  the  process  and 
finally  to  evaluate  how  far  the  student 
has  succeeded  in  achieving  his/her  goal 
As  a result  learners  should  be  able  to 
recognize  their  responsibility  in  the 
learning  process  and  to  diagnose  what 
the  next  step  is.  Also  very  important  in 
this  assessment  tool  is  that  there  is  a 
genuine  attempt  to  involve  both  student 
and  tutor  in  the  decision  making  pro- 
cess. 

At  our  program  the  Progress  Profile 
has  been  well  received  by  staff  members 
who  predict  tutors  may  be  eager  to  try  it 
out  because  of  its  compact  size  yet  fairly 


comprehensive  format.  In  the  past, 
learners  made  progress,  but  there  was  no 
systematic  mechanism  to  measure  that 
progress  and  the  process  involved. 

The  Read/Write/Now  Adult  Learning 
Center  Assessment  Adventures  portfolio  by 
Janet  Kelly  (November  1991  and  May 
1992,  Adventures  in  Assessment)  has  also 
been  well  received  at  our  program.  It 
would  be  practically  impossible  to 
absorb  all  the  different  and  valuable 
tools  that  this  center  has  created.  There- 
fore, a limited  number  of  tools  have  been 
chosen  to  be  used.  Since  we  operate  with 
volunteers  who  tutor  no  more  than  four 
hours  per  week,  it  is  necessary  to  create 
an  effective  assess-ment  system  that 
does  not  require  a tremendous  amount 
of  time  to  implement.  Our  volunteers 
work  full  time  jobs  for  the  most  part  so 
the  program  is  very  careful  not  to  take 
too  much  time  evaluating  progress  and 
to  expect  unrealistic  goals.  In  addition,  in 
the  process  of  selecting  tools  modifica- 
tions were  made  in  order  to  adapt  such 
tools  to  particular  needs  and  aspects  of 
the  program.  The  following  is  a list  of 
what  was  considered  to  be  suitable  for 
our  program:  Book  List,  Looking  at  Your 
Own  Reading  Behavior,  Reading  Confer- 
ence Record,  Teacher  Log,  Learner's  Log, 
Goals  List,  Reading  Progress  Checklist, 
and  the  Writing  Progress  Checklist. 

We  have  adapted  the  above  tools 
and  presented  them  to  tutors,  to  provide 
them  with  different  options.  In  principle, 
tutors  are  expected  to  test  them  out  and 
then  decide  what  works  for  them  given 
that  each  tutoring  situation  is  different 
from  one  another. 

Taking  into  account  that  there  might 
be  a tendency  to  not  use  the  above 
assessment  tools,  we  have  tried  to 
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The  idea  to  ask 
learners  to  think 
about  how  they 
read  seemed 
appropriate  to  us. 
It  encourages 
learners  to  think 
critically  about 
how  they  confront 
a given  text  or 
reading. 


consolidate  all  the  different  options  into 
our  current  monthly  report  that  hope- 
fully would  allow  us  to  perceive  how 
learners  are  progressing. 

Here  is  lhat  our  tool  kit  or  portfolio 
looks  like  after  adaptations  and  revi- 
sions. 

Reading  List  We  changed  Kelly's 
Book  List  to  Reading  List  because  most 
of  our  students  (both  ABE  and  ESL)  do 
not  necessarily  read  complete  books. 
Instead,  they  concentrate  on  shorter 
pieces  of  readings.  The  reasons  are 
obvious.  Reading  a complete  book  in 
class  or  for  class  would  require  a long 
period  of  time.  Nor  are  all  our  students 
able  to  read  complete  books.  Even  those 
who  can  may  still  need  a lot  of  time  to 
finish  a book.  Therefore  this  would  limit 
other  class  activities  that  are  also  neces- 
sary to  develop  and  enhance  their 
reading  and  writing  skills. 

Nevertheless,  the  rationale  was  very 
much  appreciated  among  our  staff. 
Students  do  need  to  gain  a sense  of 
accomplishment  This  is  therefore  a good 
way  to  keep  track  of  whatever  reading 
occurs  in  and  outside  of  class  as  well  as 
registering  achievements. 

Reading  Behavior  Checklist  The 
idea  to  ask  learners  to  think  about  how 
they  read  seemed  appropriate  to  us.  It 
encourages  learners  to  think  critically 
about  how  they  confront  a given  text  or 
reading.  The  questions  posed  are  essen- 
tial in  order  to  elucidate  important 
reading  issues,  for  learners  — ideally  — 
will  be  stimulated  to  discover  and 
reinvent  their  own  reading  strategies. 
Taking  into  consideration  the  level  of 
difficulty  that  some  of  these  questions 
may  have  for  some  learner  whose  read- 


ing and  writing  skills  are  not  that  devel- 
oped, it  was  decided  to  simplify  them 
and  to  incorporate  most  of  the  questions 
into  the  monthly  report  (see  the  attached 
monthly  report;  this  report  is  still  in 
process). 

Reading  Conference  Record.  Once 
again  the  basic  principle  behind  this  tool 
was  well  received.  Keeping  a record  of 
questions,  observations  and  reflections 
about  sustained  readings  enables  teach- 
ers to  understand  problems  and  suc- 
cesses learners  may  have  in  reading.  A 
minor  modification  to  this  form  had  to 
be  done  where  it  says  "BooK  Read." 
Instead  we  put  "Reading"  for  the  same 
reasons  indicated  in  the  Reading  List. 

Teacher's  Log.  This  is  an  essential 
assessment  tooL  It  allows  the  teacher  to 
maintain  a conspicuous  and  adequate 
record  of  classroom  issues.  This  way 
teachers  do  not  have  to  look  for  notes 
scattered  all  around  desks  filled  with 
papers,  etc.  The  advantage  is  that  teach- 
ers and  tutors  can  immediately  register 
difficulties,  concerns,  and  new  interests 
learners  may  have.  Nothing  has  been 
changed  in  this  tool  and  tutors  are 
strongly  encouraged  to  use  it.  Some  of 
our  staff  members  are  currently  using 
this  tool  as  welL  It  can  be  noted  that 
continual  review  and  analysis  of  these 
notes  can  lead  to  properly  addressing 
learners'  needs. 

Learner's  Log.  The  Learner's  Log  is 
another  essential  assessment  tool.  It 
helps  the  learner  maintain  a good  record 
of  what  he  or  she  views  as  improvement 
in  their  reading  and  writing  skills.  This 
is  very  important  for  teachers  because  it 
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will  help  us  see  what  students  consider 
important  indications  or  markers  of 
progress  and  what  part  of  instruction 
has  had  a significant  impact. 

In  the  same  way,  the  Learning  Diary 
by  Johan  Uvin  (November  1991)  has 
been  incorporated.  Although  it  is  similar 
to  the  previous  one,  it  was  included 
because  it  addresses  issues  that  pertain 
to  ESL  students. 

Spelling  Self  Test  This  is  not  neces- 
sarily a striking  spelling  device,  but 
important  to  the  extent  that  students  can 
test  their  spelling  on  their  own  and 
concentrate  on  words  whose  spelling 
they  need  to  review  immediately. 

Goals  List  Asking  learners  to  iden- 
tify their  goals  for  learning  is  unques- 
tionably very  important.  Currently  we 
use  a tool  that  has  been  designed  by  the 
California  Department  of  Education 
(Bureau  of  Adult  Literacy)  called 
CALPEP,  California  Adult  Learner 
Progress  Evaluation.  Although  most  of 
the  goals  listed  in  the  Read/Write  Goals 
List  are  the  same  as  CALPEP's,  there  are 
items  in  the  latter  that  could  be  incorpo- 
rated to  CALPEP,  including  Personal 
Goals:  1,2,5,8,16,17;  Family:  1;  Commu- 
nity: 2;  Work:  2, 5, 8;  and  Educational 
Goals:  all. 

The  last  two  assessment  tools  I found 
of  tremendous  value  are  the  Reading 
Progress  Checklist  and  the  Writing 
Progress  Checklist.  These  tools  were  not 
incorporated  into  the  portfolio  to  be 
used  by  tutors.  Some  of  the  points  raised 
in  them,  however,  have  been  incorpo- 
rated into  the  monthly  report.  In  addi- 
tion, I have  decided  to  use  them  for  the 


following  reasons.  First,  both  the  reading 
and  writing  checklist  allow  me  to  regis- 
ter specific  reading  and  writing  issues, 
such  as  how  well  learners  understand  in- 
class  readings,  participation,  ability  to 
select  topics  to  write  about,  etc.  Second,  I 
could  look  at  this  information  periodi- 
cally to  gain  an  understanding  of  which 
areas  learners  are  improving  in  and 
which  ones  continue  to  be  problematic. 
Overall,  what  I find  even  more  valuable 
is  that  all  the  different  assessment  tools 
tackle  different  problems  in  ways  that 
might  normally  be  overlooked.  In  this 
portfolio  there  is  an  ODDOrtunitv  to  deal 
with  every  learning  issue  in  its  own 
sphere  and  consequently  the  level  of 
learner's  success  may  increase  more 
rapidly. 

Finally/another  tool  that  attracted 
our  attention  was  the  Listening  Progress 
Tool  described  in  Letter  from  Uruguay , 
Volume  4.  This  is  an  ingenious  adapta- 
tion of  some  of  the  tools  which  appeared 
in  previous  issues  of  this  journal.  We 
found  it  to  be  quite  relevant  to  the  work 
we  do  with  the  ESL  students.  We 
thought  this  would  be  a very  good  way 
to  measure  how  ESL  students  do  in  the 
outside  world. 

We  consider  that  progress  in  reading 
and  writing  leads  to  progress  in  learners' 
ability  to  communicate  outside  the 
classroom  The  Listening  Progress  Tool 
addresses  this  issue  quite  well,  so  it  was 
decided  to  incorporate  it  into  the 
monthly  report.  As  usual  changes  have 
been  made,  but  the  basic  principle 
remains  the  same  (see  monthly  report). 

To  conclude  I want  to  add  that 
looking  back  to  evaluate  assessment 
tools  requires  a great  amount  of  effort, 
reading,  understanding,  adapting. 
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changing  and  applying  new  ideas.  Now, 
given  that  student  populations  change 
over  time  as  do  learners'  interests,  it 
seems  essential  to  evaluate  ongoing 
assessment  practices  more  often  than  is 
usually  done.  If  not  to  change,  at  least  to 
reaffirm  that  what  is  being  done  is  still 
relevant  and  applicable.  Yet,  there  is 
always  room  for  improvement  and 
change.  How  apprehensively  and  effi- 
ciently we  do  this  will  determine  the 


success  of  our  work.  Students'  progress 
will  to  some  extent  reflect  such  effort. 
With  this  in  mind,  the  Alameda  County 
Library  Adult  Literacy  Program  gave  me 
the  task  to  investigate  alternative  assess- 
ment tools  and  look  at  the  different 
options  I found  in  Adventures  in  Assess- 
ment journals.  In  this  article  I wanted  to 
share  the  result  of  this  process  and  our 
appreciation  for  the  valuable  ideas  that 
enriched  our  assessment  practices. 
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MONTHLY  TUTORING  REPORT 


Monthly  Tutoring  Report  (in  process) 


Attendance-  Croup  tutors:  Note  the  cates  your  group  met  and  who  attended.  One  to 
one  tutors:  put  the  dates  you  and  your  student  met. 

Studs  nt(e)/Tutor<e)  Dates 


Reading 

What  did  you  read  thia  month? 


Students  and  tutors:  please  answer  the  following  Questions,  /ou  may 
answer  uye*'\  "no",  “eonrnimea"  or  anything  you  think  la  correct. 

1)  Which  activities  helped  you  in  your  reading 

2)  What  did  you  Team  thia  month  about  reading 

3)  Old  you  understand  most  of  whit  you  read"  In  your  tutoring  sessions 

4)  Do  you  try"tcT understand  ^ whatyou  read  by: 

“thinking  about  the  title 
••looking  at  the  pictures 

“thinking  about  how  what  you  are  reading  relates  to  your  life 

3)  Can  you  read  and  understand  more  words  than  you  did  before? 

8)  If  you  don't  know  a word  do  you  try  to  figure  it  euTby: 

“sounding  out  the  tetters 

“breaking  it  down  Into  smaller  words  or  syllables 

“reading  the  words  around  it  and  guessing  what  words  would  make 

sense 

?)  What  continues  to  be  a problem  in  your  reading? 

Writing 

What  did  you  write  this  month 

1)  Which  activities  helped  you  In  your  writing 

2)  What  did  you  Team  this  month  about  writing 

3)  What  continues  to  be  a problem  in  your  writing? 

4)  Can  you  apell  more  words  than  you  could  last  month  or  before? 

5)  If  you  can't  spell  a word  do  you  leave  it  blank,  or  spoil  as  much  as  you 
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An  Analysis  of  Adventures  in  Assessment 

Images  of  Participatory  Assessment 
in  Adnlt  Education 


As  a sometime  adult  literacy 

practitioner  particularly  inter 
L ested  in  alternative  assessment, 
reading  through  all  five  issues  of  Adven- 
tures in  Assessment  felt  akin  to  attending 
a dream  conference.  The  forty-odd 
articles  in  these  journals,  all  written  by 
adult  literacy  practitioners  and  program 
staff,  offer  a dazzling  variety  of  assess- 
ment ideas,  tools  and  procedures,  as 
well  as  the  stories  behind  their  develop- 
ment and  ongoing  revision. 

A bi-annual  journal.  Adventures  in 
Assessment  is  edited  by  Loren  McGrail 
and  published  by  SABES,  the  Massachu- 
setts System  for  Adult  Basic  Education 
Support.  Each  edition  begins  with  an 
introduction  by  McGrail,  and  includes 
articles  written  by  practitioners  working 
in  a wide  variety  of  contexts,  including 
workplace,  ESL,  family  literacy,  ABE 
and  GED  programs.  Most  pieces  include 
samples  of  specific  forms,  checklists, 
questionnaires,  surveys  or  interview 
questions,  along  with  a narrative  ex- 
plaining the  process  of  developing, 
using  and,  often,  revising  these  tools. 

The  idea  behind  the  publication  is  that  it 
"will  become  a resource  by  and  for 
practitioners  to  select  and  adapt  tools  for 
their  own  contexts"  (McGrail,  AiA,  Vol. 
1,  p.  ii). 

The  first  three  volumes  of  Adventures 
in  Assessment  are  entitled  "Getting 
Started,"  "Ongoing"  and  "Looking  Back, 
Starting  Again,"  representing  three 
"stages"  in  learner  and  program  assess- 


ment. According  to  the  cover  page  of 
Volume  4,  "the  first  three  volumes  of 
Adventures  in  Assessment  present  a 
comprehensive  view  of  the  state  of 
practice  in  Massachusetts."  The  fourth 
volume  contains  articles  in  all  three 
categories,  and  Volume  5 (October  1993), 
called  "The  Tale  of  the  Tools,"  includes 
reflective  nieces  written  bv  practitioners 

X + A 

who  were  involved  in  Component  Three 
of  the  Greater  Opportunities  in  Adult 
Learner  Success  (G.O.A.L.S)  Project 
developed  by  the  Massachusetts  Bureau 
of  Adult  Education.  Although  the  origi- 
nal intention  of  the  G.O.A.L.S.  Project 
was  to  "investigate  and  design  an  ac- 
countability system  which  would  be  a 
true  reflection  of  the  field"  (Whiton, 

AiA,  Vol.  5,  p.  10),  these  articles  demon- 
strate, instead,  practitioners' 
commitment  to  developing  alternative 
assessments  whose  purpose  is  primarily 
informing  learners  and  instruction  rather 
than  funders.  The  result  is  an  incredibly 
rich  collection  of  "tales"  of  alternative 
assessment. 

What  makes  Adventures  in  Assessment 
especially  intriguing  and  useful  is  the 
journal's  unifying  purpose,  which,  as 
McGrail  writes  in  the  introduction  to 
Volume  One,  is  "to  explore  participatory 
assessment  or  learner-centered  ap- 
proaches to  assessment  and  evaluation" 
(McGrail,  AiA,  Vol.  1,  p.  ii).  Citing  Lytle 
(1991),  McGrail  identifies  key  features  of 
participatory  adult  education  and  assess- 
ment. "The  most  important  principle," 
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she  writes,  "is  that  assessment  be  done 
with  the  learner  and  not  to  the  learner" 
(p.  ii).  Specifically,  "[pjartidpatory 
assessment  means  a collaborative  rela- 
tionship among  learners  and  program 
staff  in  determining  the  goals,  texts  and 
contexts  of  assessment,  as  well  as  in 
judging  its  outcomes"  (p.  ii). 

This  focus  on  participatory  assess- 
ment reflects  a wider  paradigm  shift  in 
adult  education.  McGrail  begins  her 
introduction  to  Volume  One  with  a 
quote  from  Lytle  (1991)  which  reveals 
the  interconnected  aspects  of  this  shift: 

Constructing  new  images  of  adults — 
images  built  on  assumptions  of  dignity 
and  competence,  of  literacy  as  reflec- 
tive and  self-critical  practice,  and  of 
learning  as  partidpatory — requires 
that  we  rethink  or  reconceptualize  not 
only  our  notions  of  what  counts  as 
literacy  but  also  our  methods  of 
inquiry — the  processes  we  use  to 
document  and  assess  learning  (Cited 
in  McGrail,  1991,  p.  ii). 

The  practitioners  in  Adventures  in  Assess- 
ment recognize  that  traditional  methods 
of  assessing  learners  and  programs 
cannot  support  their  commitment  to 
these  new  ways  of  thinking  about  adult 
learners  and  literacy  learning.  Instead, 
the  participatory  images  of  assessment 
they  present  move  towards  a redefini- 
tion of  assessment  as  learning.  Lytle  and 
Wolfe  (1989)  write  that  "[ajlthough 
various  approaches  assess  literacy  as 
skills,  tasks  and  practices,  only  participa- 
tory approaches  have  the  potential  for 
assessing  literacy  as  critical  reflection" 

(p.  58).  Perhaps  this  is  because  taking  an 
active  role  in  assessing  oneself  and  one's 
world  results  in  the  "internal  change  of 
consciousness"  which  Brookfield  argues 


is  central  to  self-directed  learning:  "This 
consciousness  involves  an  appreciation 
of  the  contextuality  of  knowledge  and  an 
awareness  of  the  culturally  constructed 
form  of  value  frameworks,  belief  sys- 
tems and  moral  codes  that  influence 
behavior  and  the  creation  of  social 
structures"  (Cited  in  Robishaw,  AiA, 

VoL  5,  p.  94).  There  is  a cyclical  relation- 
ship between  action  and  reflection,  and 
taking  an  active  role  in  assessment  is 
part  of  that  cycle:  "In  the  course  of  this 
recurring  cycle  of  action  and  reflection, 
according  to  Brookfield,  learners  become 
more  proactive,  assume  control  over 
goal  setting,  and  determine  personally 
meaningful  criteria  for  evaluating  their 
learning"  (Lytle,  1991,  p.  118).  Thus,  a 
definition  of  literacy  as  critical  reflection 
leads  to  a reconceptualization  of  assess- 
ment as  participatory  and  as  a part  of 
learning.  And,  similarly,  reports  of 
"experiments"  with  more  participatory 
assessment  practices  may  help  practitio- 
ners, learners  and  researchers  better 
understand  the  nature  of  critical  reflec- 
tion in  adult  education. 

Recognizing  the  centrality  of  the 
active  involvement  of  adult  learners  in 
participatory  assessment  practices, 
McGrail  and  the  practitioners  contribut- 
ing to  Adventures  in  Assessment  seek  to 
reconceptualize  assessment  by  upsetting 
the  traditional  roles  of  "teachers"  and 
students."  In  particular,  they  make  a 
commitment  to  look  for  ways  to  involve 
adult  learners  in  the  development,  use 
and  revision  of  the  tools  and  processes 
used  to  assess  learner  progress  and  to 
evaluate  adult  literacy  classes  and 
programs.  What  does  it  look  like  when 
adult  literacy  practitioners  attempt  to 
make  assessment  practices  partidpa- 
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tory?  What  new  images  of  assessment 
and  possibilities  for  learner  participation 
emerge  in  these  articles,  and  what 
questions  and  issues  do  they  raise? 

Possibilities  for  Participatory 
Assessment  Practices 

Figure  1 illustrates  some  examples  of 
the  roles  learners  are  asked  to  take  on  in 
the  assessment  practices  described  in 
Adventures  in  Assessment.  These  practices 
can  be  categorized  as  'initial',  'ongoing' 
and  looking  back'  assessments  of 
learner  progress  (McGrail,  AiA,  Vol.  1), 
assessments  of  classes  and  programs, 
and  learner  participation  in  the  develop- 
ment and  revision  of  assessments.  To 
help  illuminate  the  participatory  aspects 
of  these  practices  and  some  of  the  ques- 
tions they  raise,  I will  relate  them  to 
three  of  tire  "general  features"  of  learner- 
centered  or  participatory  assessment 
practices  that  Lytle  observes  emerging 
from  other  "grassroots  research  and  staff 
development  projects"  (Lytle,  1988,  p.  3): 

(1)  Adults  are  active  participants,  co- 
investigators in  determining  and  describing 
their  own  literacy  practices,  strengths  and 
strategies.  Whether  initiated  by  an  adminis- 
trator, teacher/tutor,  or  by  adults  themselves, 
the  design  and  implementation  of  the  proce- 
dures constitute  a dialogue  or  collaboration. 
(Lytle,  1988,  p.  3) 

The  initial  assessment  practices 
described  in  Figure  1 (next  two  pages) 
illustrate  adults  co-investigating  their 
own  literacies  through  discussion, 
reading  and  writing.  In  contrast  to  the 
traditional  role  of  test  taker  and  recipient 
of  expert  diagnoses,  these  adult  learners 
are  asked  to  generate  or  choose  their 


own  goals,  answer  questions  based  on 
their  own  knowledge  of  their  abilities 
and  interests,  and  even  "place"  them- 
selves into  classes  through  a process  of 
self  evaluation  and  discussion  with  staff. 
Rosen  (AiA,  Vol.  2)  writes  about  a 
portfolio  assessment  process  that  asks 
students  to  choose  not  only  the  contexts, 
but  also  the  purpose  and  audience  for 
their  portfolio;  this  illustrates  a practice 
that  encourages  adult  learners  to  de- 
scribe themselves  in  their  own  terms. 

Several  practitioners  raise  concerns 
about  what  it  might  mean  to  ask  learners 
to  describe  their  progress,-  however. 
Isserlis,  working  with  ESL  learners, 
worries  that  "[f]or  learners  with  little 
prior  schooling  or  from  cultures  where 
teachers  dictate  what  happens  in  a 
classroom,  the  concept  of  self-assessment 
may  be  difficult  to  grasp.  Surely,  many 
learners  have  an  innate  sense  of  their 
own  movement  with  language  and 
literacy,  but  the  expression/  verbaliza- 
tion of  that  progress  may  not  be  within 
the  frameworks  they  have  developed  or 
use  in  describing  learning"  (AiA,  Vol.  2. 
p.  6).  Isserlis  supports  her  concern  with 
examples  of  learners'  very  general 
answers  to  survey  questions  such  as 
"What  do  you  think  you  learned  this 
year?"  Often,  she  believes,  the  answers 
to  such  questions  (e.g.  "Before  I under- 
stand nothing,  now  I understand  small") 
reflect  learners'  modesty,  cultural  back- 
ground, or  desire  to  show  appreciation 
to  the  teacher,  rather  than  revealing  an 
awareness  of  specific  progress.  While, 
Isserlis  writes,  "there  is  value  in  the 
process  of  asking  questions  about  how 
learners  learn  and  about  how  they  feel 
they  are  progressing...[t]his  talk  about 
leaming...is  somewhat  of  a language  of 
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FIGURE  Is 

SOME  EXAMPLES  OF  PARTICIPATORY  ASSESSMENT  PRACTICES 
FROM  ADVENTURES  IN  ASSESSMENT 

Figure  1 

Some  Examples  of  Participatory  Assessment  Practices 
from  Adventures  in  Assessment 

Part  1:  Learner  Self  Assessment 


Assessment  Practice 

Learner  Role  in  Assessment 

Article 

INITIAL 

• Pre-goal  setting  group 
reflection 

Reflect  as  a group  on  past  schooling  and 
life  experiences;  create  timeline  of  past 
and  future  life  events. 

Urban  Corps 
Expansion 
Proj.  Vol.  4 

• Goal  setting 

Generate  goals  in  answer  to  open-ended 
written  or  oral  questions,  select  goals 
from  a "Goals  List."  or  prioritize  goals 
generated  as  a group. 

Kelly,  Vol.  1 
Ebbit,  et  a! 
Vol.  2 

* Reading  Assessment 

Choose  learner  writing  to  read  and  discuss 
Answer  questions  such  as  "Does  this 
reading  level  feel  too  hard  or  too  easy?" 

Barry 
Vol.' 4 

* Class  placement 

Complete  learning  questionnaires, 
a seif-evaluatory  math  sheet,  self- 
selected  reading/writing  activities,  and 
discussion  with  staff  - then  "place"  self. 

Trunnel 
Vol.  2 

ONGOING 

• Shared  teacher  logs 

Read  and  write  in  shared  observation  log 
Use  teacher  observations  as  data  for 
self-reflection 

Kellv 
Vol.  5 

• Learning  Logs 

Keep  record  of  what  accomplished 
Record  frustrations,  questions,  goals 
See  what  you  know  thru  writing 

Trunnel 
Vol.  5 

Gear  Vol.  5 

• Writing  Process 
Checklist 

Fill  in  checklist  with  teacher  to  record 
writing  strategies  used 

Kelly 
Vol.  2 

LOOKING  BACK 

• Portfolios 

Choose  audience,  purpose,  design  for 
portfolio  to  reflect  learning 

Rosen 
Vol.  2 

Assessment  Practice 

Learner  Role  in  Assessment 

Article 
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FIGURE  It 

SOME  EXAMPLES  OF  PARTICIPATORY  ASSESSMENT  PRACTICES 
FROM  ADVENTURES  IN  ASSESSMENT 


Figure  1.  Cont.'d 

Some  Examples  of  Participatory  Assessment  Practices 
from  Adventures  in  Assessment 

Part  2:  Learner  Assessment  of  Classes  & Programs,  and 
Learner  Participation  in  Developing  & Revising  Assessment 
Practices 


Assessment  Practice 

Learner  Role  in  Assessment 

Article 

CLASS  EVALUATION 

• Weekly  and  End  of  Cycle 
Written  Evaluations 

List  class*  weekly  activities  and  rate  them 
Answer  questions  such  as  “Which 
activities  were  most  useful  and  why?** 

For  final  evaluation,  answer  questions 
such  as  Tf  you  were  the  instructor,  what 
would  vou  do  differently?" 

Gear 
Vol.  5 

PROGRAM 

EVALUATION 

• Town  Meeting 

In  groups,  give  feedback  & suggestions  to 
program  staff. 

Cason 
Vol.  3 

• Questionnaire 

Students  on  Advisory'  Board  create 
questionnaire  w/staff  for  other  students 
about  the  program.  All  student  responses 
are  considered  in  action  plan  developed  by 
staff. 

Reddy 
Vol.  3 

DEVELOPMENT  OF 

ASSESSMENT 

PRACTICES 

• Creation  of  Organizational 
Assessment  System  for  New 
Program 

Serve  on  Advisory  Board  with  staff  & 
share  assessment  preferences  as  pan  of 
initial  intake  process. 

Uvin 
Vol.  I; 

REVISION  OF 
ASSESSMENT 
PRACTICES 

• Individual  Feedback  on 
Assessment  Practices 

One  student  discusses  value  of  specific 
assessment  practices  to  her  with  teacher 

Barry 
Vol.  5 

Volume  6:  Spring  1994 


ADVENTURES  IN  ASSESSMENT 


so 


Responding 
to  the  Dream 
Conference 


Perhaps  the 
process  of  working 
together  to  learn  a 
' language ' with 
which  to  assess 
ourselves  is 
inseparable  from 
'growth  and 
change.' 


its  own  and  must  be  learned  as  (yet 
another)  language"  (p.  8).  Isserlis' 
comments  focus  practitioners  again  on  a 
central  question:  Who  is  assessment  for? 
One  hope  is  that  learning  the  kind  of 
"metalanguage"  needed  to  talk  about 
changes  in  their  reading/ writing  strate- 
gies, for  example,  is  part  of  a more 
general  process  of  critical  reflection  and 
growth.  McGrail  points  out  that  this 
might  be  true  for  everyone: 


All  of  us — administrators,  counselors, 
teachers  and  students — find  it  difficult 
to  talk  about  literacy  and  language 
development  beyond  talking  about 
skills.  We  are  all  new  to  thinking  about 
learning  as  a process  and  not  just  a 
product.  What  Isserlis  claims  is  true 
for  many  learners...  is  probably  true 
for  practitioners  as  welL  We  are  just 
beginning  to  develop  our  own  frame- 
work and  don't  always  know  how  to 
translate  our  thoughts,  hunches,  and 
ideas  about  what  constitutes  progress 
into  practice,  tools  and  procedures  that 
measure  and  document  what  we 
believe  to  be  real  indicators  of  growth 
and  change"  (AiA,  Vol.  3,  p.  vi). 


Perhaps  the  process  of  working  together 
to  leant  a language'  with  which  to 
assess  ourselves  is  inseparable  from 
'growth  and  change.'  One  caveat  to  this, 
however,  is  that  we  need  to  make  sure 
that  learners  are  not  stifled  or  silenced  in 
this  process.  This  might  mean  encourag- 
ing learners  to  use  their  first  language  or 
other,  nonverbal  modes  of  communica- 
tion as  they  take  part  in  this  process  of 
assessment/literacy  learning.  The 
overarching  question  we  are  left  with  is 
one  of  communication  and  collaboration: 
In  what  ways  can  practitioners  and 
learners  talk  and  work  together  to  build 
frameworks  for  critical  reflection  and 
assessment? 


Practitioner/ authors  in  Adventures  in 
Assessment  report  less  involvement  of 
learners  in  the  design  and  revision  of  the 
assessment  practices  they  discuss  than  in 
their  implementation.  While  most  of  the 
authors  relate  that  they  revise  assess- 
ment practices  based  on  learners' 
reactions  to  them  in  the  classroom,  only 
a few  report  involving  learners  in  the 
initial  design  of  assessment  practices 
(Uvin,  AiA,  Vol.  1)  or  asking  for  explicit 
feedback  on  the  usefulness  of  particular 
practices  to  learners  (Barry,  AiA,  Vol.  5). 
In  Volume  5,  however,  several  authors 
look  closely  at  what  assessment  practices 
have  taught  them  about  particular 
learners  and  find  themselves  wanting  to 
know  more  about  "how  learners  experi- 
ence learning"  (McGrail,  AiA,  Vol.  5, 
p.5).  Trunnel,  for  example,  investigates 
the  contents  of  two  students'  portfolios 
and  evaluates  the  usefulness  of  various 
assessment  tools.  He  realizes,  though, 
that  he  cannot  do  this  alone:  "All  this  is 
fine.  But  what  is  helpful  FOR  THE 
STUDENT?"  (AiA,  VoL  5,  p.  46).  In  her 
piece  (co-authored  by  Pat  F.,  her  stu- 
dent), Barry  (AiA,  Vol.  5)  asks  Pat  which 
assessment  tools  she  finds  useful. 

Barry's  report  of  their  discussion  repre- 
sents a needed  movement  towards 
learner  involvement  in  designing, 
revising,  and  reporting  on  new  assess- 
ment practices  and,  ultimately,  in 
building  new  frameworks. 

(2)  Rather  than  adhering  too  strictly  to  a 
predetermined  script,  learner-centered 
assessment  involves  dynamic  exchanges 
among  learners,  texts  and  teachers/tutors. 
Wien  difficulties  are  encountered,  assistance 
is  given  rather  than  vnthheld ; the  social 
situation  provides  a supportive  context  for 
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experimentation  and  risk-taking  (Lytle, 

1988,  p.3). 

Practices  such  as  asking  learners  to 
choose  which  texts  to  read  during  initial 
placement  interviews  contradict  the 
rigidity  of  traditional  tests.  Other  prac- 
tices that  illustrate  'dynamic  exchanges' 
include  open-ended  self  and  course 
evaluation  questions,  'town  meetings' 
(Cason,  AiA,  Vol.  3)  and  shared  teacher 
logs  (Kelly,  AiA,  VoL  5).  The  idea  of  a 
practitioner  sharing  the  anecdotal 
records  she  keeps  about  students  repre- 
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roles;  rather  than  being  the  evaluator  of 
students'  progress,  the  practitioner  who 
shares  her  observations  about  students 
becomes  a co-investigator,  and  her 
students  can  use  her  perspective  as  data 
for  self-reflection.  This  would  also  seem 
to  change  the  power  relationships  in  the 
classroom,  contributing  to  a "supportive 
environment  for  experimentation." 

While  only  one  practitioner  reports 
having  worked  on  ways  to  share  teacher 
log  entries  with  students  (Kelly,  AiA, 
VoL  5,  p.  20)  several  others  note  in  their 
articles  that  they  are  interested  in  this 
idea. 

Asking  learners  for  feedback  about 
programs  and  classes  and  then  acting  on 
that  feedback  is  another  practice  that 
supports  open  exchanges  and  risk- 
taking on  the  part  of  both  learners  and 
practitioners.  One  inspiring  example  of 
this  is  Cason's  report  of  the  all  program 
'town  meetings'  held  by  the  ESL  learners 
and  practitioners  at  the  Log  School  in 
Dorchester,  Mass.  Teachers  provided 
questions  as  a springboard  for  two 
group  discussions  which  resulted  in 
concrete  changes  in  the  program.  These 


changes  included  adding  drop-in  tutor- 
ing times,  instituting  a different 
attendance  policy  and  including  current 
students  in  orientation  programs  for 
incoming  learners  (Cason,  AiA,  VoL  2,  p. 
13).  Less  tangible  results  of  the  meetings, 
according  to  Cason,  included  building 
the  program's  sense  of  community,  and 
involving  students  in  leadership  roles 
outside  of  the  classroom.  In  addition, 
Cason  believes  that  the  meetings  helped 
learners  envision  the  ESL  program  as  a 
part  of  a larger  community,  and  that  this 
new  vision  encouraged  them  to  see 

rAmmimiftr  icciioc  citrV»  <ac  r\oi rrV\Kr»r- 

WU  UA  IU1  UkJ  M/4/M.W4//  k/UVAt  Wk, 

hood  safety,  as  'legitimate  topics  of 
action  and  discussion  in  ESL  class" 

(p.  14).  The  open-ended  dynamics  of 
the  town  meetings  seem  to  have  been  a 
springboard  for  further  action  for  both 
learners  and  practitioners. 

(3)  What's  assessed  reflects  the  particular 
goals  of  learners  and  often  includes  (a) 
literacy  practices  in  everyday  life  (how 
adults  are  using  what  they've  learned  and 
what  significance  these  things  have  in  their 
lives),  (b)  varieties  of  tasks  and  strategies  for 
reading  and  writing  particular  texts  in 
specific  contexts,  as  well  as  (c)  learners' 
perceptions  or  theories  of  reading  and 
writing  (Lytle,  1988,  p.4). 

I will  discuss  the  implications  of 
asking  learners  to  set  goals  later  in  this 
paper;  in  terms  of  the  range  of  what  is 
being  assessed  by  the  practices  docu- 
mented here,  however,  many  of  the  goal 
setting  questions  and  activities  described 
in  Adventures  in  Assessment  do  ask 
learners  to  think  about  the  ways  they  use 
literacy  in  their  lives  and  about  the  ways 
they  think  about  reading  and  writing. 
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Along  with 
assessing 
what  learners 
accomplish  in 
relation  to  their 
language/literacy 
goals, 

practitioners 
express  interest 
in  assessing  other 
aspects  of  growth, 
including  changes 
in  learner 
self-esteem, 
confidence,  and 
community 
building. 


For  example,  Ebbit,  et  al.  (AiA,  Vol.  2) 
describe  a goal  setting  process  during 
which  "students  prioritize,  in  pairs  or  as 
a group,  the  language  survival  areas 
they  would  like  to  explore  in  their  ESL 
class"  (p.  50).  The  Read/Write/Now 
placement  interview  form  asks  adult 
learners  to  consider  their  most  important 
reasons  for  wanting  to  learn  to  read  and 
write  (AiA,  Vol.  1,  p.  20),  and  their 
questionnaire,  "Looking  at  Your  Own 
Reading  Behavior"  asks  learners  to 
answer  questions  such  as  "Do  you  ask 
yourself  questions  when  you  read?" 
(Appendix  10). 

Along  with  assessing  what  learners 
accomplish  in  relation  to  their  language/ 
literacy  goals,  practitioners  express 
interest  in  assessing  other  aspects  of 
growth/  including  changes  in  learner 
self-esteem,  confidence,  and  community 
building.  This  interest  seems  to  spring 
from  an  expanded  notion  of  what  consti- 
tutes growth  and  learning  and  from  a 
desire  to  help  learners  see  the  progress 
they  are  making  in  all  of  these  facets  of 
their  lives.  While  these  aspects  of  growth 
are  not  generally  ones  that  are  captured 
in  learners'  initial  written  goals,  practi- 
tioners report  that  these  kinds  of  changes 
are  often  the  ones  that  learners  them- 
selves notice  and  value  when  they  occur. 

Barry,  for  example,  writes  that  her 
student  Pat  "highlighted  the  value  of 
teaching  and  learning  from  [classmates], 
and  developing  a sense  of  responsibility 
towards  each  other"  (AiA,  Vol.  5, 80). 
Discussing  Pat's  enthusiastic  comments 
about  having  her  work  published  and 
offering  a workshop  to  other  teachers 
and  students,  Barry  writes,  "The  impor- 
tance of  recording  these  kinds  of 
comments  is  obvious  to  me.  This  is  the 


data  which  reflects  development  of  self- 
esteem and  self  doubt.  It  is  my  hope  that 
by  recording  segments  of  conversation 
with  students,  over  time,  I will  gather  a 
true  reflection  of  their  views  of  them- 
selves as  learners"  (p.  79). 

She  then  goes  on  to  write  that  she 
has  heard  of  another  teacher  who  shared 
this  data  with  learners  through  her 
teachers  log,  and  comments,  "I  hope  to 
make  my  log  available  so  that,  just  as 
with  the  other  assessment  tools,  the  log 
is  not  a device  for  a teacher  to  measure  a 
student's  progress,  but  a method 
through  which  learners  can  assess  their 
own  growth."(p.  79).  Barry's  thoughts 
on  the  importance  of  sharing  the  infor- 
mation she  collects  about  learners 
emphasize  that  the  primary  purpose  of 
this  kind  of  information  should  be  to 
inform  learner  self-assessment  and  not 
to  convince  funders  of  a program's 
viablity;  uses  of  this  kind  of  information 
which  do  not  include  the  learner  bypass 
the  opportunity  for  critical  reflection  and 
therefore  seem  unproductive,  if  not 
exploitative. 

This  conclusion  raises  the  question, 
though,  of  what  kind  of  information 
about  learner  progress  should  be  used  for 
accountability  purposes.  Comings,  in  a 
letter  in  Vol.  3 of  Adventures  in  Assess- 
ment, argues  that  learner  assessment 
should  not  be  used  for  program  evalua- 
tion: 

Using  student  assessment  as  die 
measure  of  effectiveness  for  program 
accountability,  no  matter  how  good 
die  assessment  tool,  will  always  make 
die  test  result  die  focus  of  programs, 
rather  than  die  needs  of  die  student. 
Funding  agencies  do  have  a legitimate 
right  to  measure  die  effectiveness  of 
the  programs  they  fund.  But,  looking 
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at  student  progress  does  not  necessar- 
ily provide  a way  to  judge  whether  or 

not  money  is  being  well  spent"  (p.  43). 

Comings  argues  instead  that  programs 
should  be  judged  against  "standards  of 
practice  and  service, " following  the 
accreditation  model  used  by  colleges 
and  universities  (p.  44).  Such  a model 
might  free  practitioners  and  learners  to 
focus  less  on  measuring  and  document- 
ing learner  progress  for  others  and  more 
on  making  sure  learners  have  opportuni- 
ties for  self  assessment  and  critical 
reflection. 

Overall,  the  participatory  assessment 
practices  described  in  Adventures  in 
Assessment  demonstrate  a range  of 
learner  roles  that  are  characterized  by 
action,  reflection  and  decision-making. 
They  involve  learners  in  generating 
goals,  making  choices,  assessing  their 
own  progress,  providing  feedback  and 
bringing  about  change. 

The  image  of  learners  that  informs 
and  results  from  these  practices  is  very 
different  from  the  image  inspired  by 
traditional  assessments  that  focus  on 
uncovering  deficits  and  prescribing 
remediation.  Instead  of  the  picture  of  a 
passive  adult  learner  who  waits  for 
someone  else  to  tell  her  what  she  needs 
to  learn  and  then  "gives"  her  that 
needed  knowledge,  these  assessment 
practices  paint  portraits  of  adults  who 
come  to  programs  with  valuable  experi- 
ence and  knowledge  and  with  their  own 
agendas.  As  Lytle  (1988)  writes,  "these 
new  approaches  to  assessment  commu- 
nicate respect  for  adults  — for  what  they 
bring  to  learning  and  for  what  they  come 
to  learn"  (p.  3). 

The  practitioners  writing  about  these 
new  approaches  raise  many  important 


questions  about  their  own  assumptions 
and  about  their  hopes  and  fears  for  the 
future  of  participatory  adult  education 
and  assessment.  Reading  through  these 
journals,  I hear  a conversation  among 
practitioners,  one  that  often  focuses  on 
what  to  do  next.  In  order  to  further 
clarify  this  conversation  and  to  contrib- 
ute some  ideas  towards  next  steps,  I will 
discuss  one  assessment  practice  that 
many  of  these  practitioner/ authors 
write  about:  asking  learners  to  set  goals. 

Learners  Setting  Goals 

One  nossibilitv  for  learner  oarticiDa- 

JT * * * 

tion  in  initial  and  ongoing  assessment 
that  is  advocated  by  almost  every  author 
is  learner  goal  setting.  According  to 
McGrail,  a key  principle  of  alternative 
assessment  is  that  "it  should  help  the 
learner  achieve  his  or  her  goals.  In  other 
words,  what  is  assessed  must  reflect 
what  the  learner  wishes  or  needs  to 
accomplish"  (AiA,  Vol  1,  p.  3).  This 
assumption  leads  to  practices  designed 
to  help  learners  uncover  and  articulate 
their  initial  and  ongoing  goals.  In  an  ESL 
class,  this  might  take  the  form  of  learners 
drawing  maps  to  show  where  want  to  be 
able  to  use  English  (Fandel,  AiA,  Vol.  1). 
In  a class  for  adult  beginning  readers 
and  writers,  it  might  involve  checking 
off  goals  from  a Goals  List  created  by 
practitioners  (often  collected  from  past 
learners)  or  answering  more  open-ended 
questions  such  as  "What  are  some  things 
that  you  want  to  do  that  being  able  to 
read  and  write  better  will  help  you  do?" 
(Kelly,  AiA,  Vol.  1).  Some  programs  ask 
learners  to  generate  or  choose  these 
goals  independently,  while  others  set  up 
group  discussion  or  conversations  with 
teachers  for  the  purpose  of  setting  goals. 
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Learner  goals 
are  usually 
"revisited"  both 
during  and  at  the 
end  of  class  cycles; 
clearly,  many 
practitioners  see 
learner  involve- 
ment in  setting, 
meeting  and 
revising  goals 
as  a valuable 
assessment 
practice. 


A variety  of  goals  are  suggested  by 
the  goals  lists;  these  are  often  catego- 
rized under  titles  such  as  "Personal," 
"Family,"  "Work,"  "Community,"  and 
"Academic"  (see  Vol.  1,  appendices  8, 15, 
18  & 20).  Such  categories  reflect  a defini- 
tion of  literacy  as  practices  and  an 
understanding  that  adults'  literacies  are 
multiple  and  context-specific.  Examples 
of  goals  from  one  checklist  include:  'To 
read  a phone  book",  'To  write  notes  to 
school",  'To  register  to  vote",  "Punctua- 
tion", and  'To  tell  time"  (Germanowski, 
AiA,  Vol.  1,  Appendix  17).  Illustrative  of 
the  few  learner  goal  statements  that  were 
included  in  articles  (most  presented 
blank  rather  than  completed  forms)  are 
statements  such  as  "Understanding 
bigger  words,"  'To  write  letters  to 
friends  and  family,"  and  "I  will  like  can 
speak  English  to  be  a beautician" 
(Fandel). 

In  many  classes,  learners  are  asked  to 
record  their  goals  and  then  monitor  their 
progress  towards  meeting  them.  Some 
programs,  such  as  Read/Write/Now  (a 
library-sponsored  program  for  adult 
beginning  readers  and  writers)  also  use 
learning  contracts'  which  involve 
learners  in  creating  a "plan  of  action"  for 
meeting  their  stated  or  chosen  goals 
(Kelly,  AiA,  Vol.  1).  Learner  goals  are 
usually  "revisited"  both  during  and  at 
the  end  of  class  cycles;  clearly,  many 
practitioners  see  learner  involvement  in 
setting,  meeting  and  revising  goals  as  a 
valuable  assessment  practice. 

Indeed,  these  practitioners  consider 
learner  goal  setting  to  be  central  to 
participatory  education.  Goal  setting  is 
seen  as  empowering  because  it  asks 
learners  to  identify  what  they  want  to 
accomplish  and  motivating  because  it 


encourages  them  to  focus  on  the 
progress  they  make  towards  their  own 
goals.  Kelly  articulates  this  belief  in  the 
context  of  the  Read/Write/Now  pro- 
gram: 

One  of  the  goals  of  a whole-language 
program  is  to  empower  learners  by 
helping  them  to  become  more  self- 
directed,  to  identify  and  work  towards 
their  own  goals  for  learning,  literacy 
and  life...  Both  learners  and  teachers 
need  to  know  why  they  do  what  they 
do  in  a classroom  so  that  they  can  have 
a sense  of  progress,  as  well  as  make 
decisions  about  future  directions. 

(AiA,  Vol.  1,  p.17) 

As  Kelly's  words  imply,  practitioners 
also  attempt  to  change  power  relation- 
ships by  basing  curriculum  and 
instruction  on  the  goals  that  learners  set. 
For  example,  in  a written  'conversation' 
with  McGrail,  Lindy  Whiton,  coordina- 
tor of  Component  Three  (alternative 
assessment)  of  the  G.O.A.L.S.  Project, 
describes  the  relationship  between  goals 
and  curriculum  at  the  Log  School  (see 
also  Cason,  AiA,  Vol.  3):  "When  they  get 
a group  of  people  in,  they  take  those 
goals.. .that  are  the  most  in  common 
across  all  learners  [and  that]  is  where 
they  start  their  curriculum... .In  the  end 
the  students  do  the  evaluation:  Did  your 
goals  get  met?"  (McGrail  and  Whiton, 
AiA,  VoL  3,  p.  39-40).  Many  practitioners 
describe  more  individualized  processes 
for  using  learners'  goals  to  develop 
curriculum;  these  might  involve  using 
goal  sheets  to  create  specific  assignments 
(Gluckman,  et.  al.,  AIA,  Vol.  3,  p.  26)  or 
activity  plans  (Martin,  Hall  & Bahre, 
AIA,  VoL  4,  p.  16).  One  hope  is  that  this 
kind  of  learner  input  will  result  in  a 
curriculum  that  directly  addresses 
learners'  lives  and  interests. 
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In  addition  to  providing  learners 
with  inspiration  and  with  input,  practi- 
tioners regard  goal  setting  as  a way  to 
help  learners  see  learning  as  active  and 
themselves  as  subjects:  "It  is  impossible 
to  see  education  as  the  passive  receiving 
of  information  from  others  when,  as  a 
learner,  you  have  just  written  down  a 
plan  of  action  for  meeting  your  own 
goals  in  reading  and  writing,  and  this 
plan  involves  you  in  doing  things,  not 
just  listening  while  a teacher  tells  you 
about  doing  things"  (Kelly,  AiA,  Vol.  1, 
p.  26).  These  practitioners'  conceptions 
and  uses  of  learner  goal  setting  as  an 
integral  part  of  assessment  reinforce  a 
definition  of  learning  as  participatory 
and  of  literacy  as  practices  and  critical 
reflection. 

However,  some  practitioners/ 
authors  also  express  concerns  about  the 
practice  of  asking  adult  learners  to  set 
goals,  concerns  centered  on  the  need  for 
cultural  sensitivity  and  for  dialogue  and 
negotiation.  For  example,  Don  Robishaw 
(a  SA6ES  Research  Consultant)  sees  goal 
setting  as  "a  linear,  future-oriented, 
individualistic.  Western  phenomenon 
which  is  not  always  transferable  to 
English  as  a Second  Language  (ESL) 
learners  from  non-Westem  cultures  and 
other  domestic  cultural  contexts"  (AiA, 
VoL  5,  p.  93).  Writing  from  the  perspec- 
tive of  someone  who  "grew  up  in  a 
housing  project"  and  "avoided  schooling 
as  much  as  possible  up  to  the  age  of 
sixteen,"  (p.  93),  Robishaw  worries  that 
goal  setting  may  be  a culturally-spedfic 
activity  that  makes  little  sense  to  learn- 
ers who  did  not  necessarily  experience 
life  as  that  'controllable'  growing  up. 

One  assumption  that  informs  participa- 
tory education  is  that  many  adult 


literacy  learners  come  to  adult  education 
having  internalized  negative  images  of 
themselves  as  learners  (Lytle,  1991); 
Robishaw  sees  this  as  both  the  problem 
with  goal  setting  — because  these  adults 
don't  necessarily  believe  that  they  can 
learn  — and  as  a possible  solution.  He 
refers  to  the  article,  "Group  Goal  Setting 
Activities:  An  Approach  from  Youth 
Service  Corps"  (AiA,  Vol.  4),  which 
describes  structured  pre-goal  setting 
discussions  in  which  youth  service 
corpsmembers  critically  reflect  together 
on  their  past  schooling  and  life  experi- 
ences. Robishaw  sees  this  participatory 
group  process  as  invaluable:  "Adult 
learners  need  to  reflect  on  their  past  life 
history  and  to  sort  through  their  experi- 
ences in  order  to  see  beyond  their  own 
formal  schooling  experiences  in  order  to 
better  develop,  persist  and  continue  with 
learning"  (p.  94-95).  Robishaw  puts  goal 
setting  in  the  larger  context  of  self- 
directed  learning  and  reminds  us  that 
learners'  experiences  need  to  be  both  the 
starting  place  and  the  medium  of  a 
participatory  assessment  process.  He 
also  demonstrates  that  goal  setting  needs 
to  be  an  interactive  process,  that  it  is  not 
enough  to  simply  ask  learners  set  goals. 

Kelly,  writing  about  the  evolution  of 
Read/Write/Now's  initial  assessment 
tools  and  processes,  echoes  this  idea. 
Citing  learner  and  practitioner  frustra- 
tion with  unrealistic  or  very  general 
goals,  Kelly  writes  about  the  need  for 
teacher  participation  in  the  goal  setting 
process: 

Learning  about  helping  learners  to 
choose  goals  and  trying  to  empower 
them  as  decision-makers  in  their  own 
learning  has  been  an  evolutionary 
process....We  have  gone  from  a stance 


In  addition  to 
providing  learners 
with  inspiration 
and  with  input, 
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and  themselves  as 
subjects. 
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of  very  limited  interference  and 
influence  in  the  learners'  decision- 
making  about  their  educational  goals 
to  the  role  of  full  participants  in  a 
learning  community.  Full  participation 
means  listening  to  each  other,  sharing 
our  opinions,  knowledge,  and  advice 
in  the  process  of  negotiating  the 
decisions  that  we  often  make  together 
(KeUy,  AiA,  Vol  1,  p.  28). 

What  Kelly  is  talking  about  is  the  bal- 
ance of  power  that  seems  to  be  at  the 
heart  of  many  practitioners'  concerns 
and  questions  about  participatory 
assessment.  It  is  important  to  recognize 
and  acknowledge  the  fact  that  most  (if 
not  all)  practitioners  and  learners  come 
to  adult  education  with  images  of 
"teachers,"  "students"  and  "school"  that 
involve  unequal  power  relationships. 
Participatory  education  and  assessment 
is  about  trying  to  change  these  relation- 
ships, and  this  is  a very  difficult  task.  In 
a fascinating  way,  the  evolution  Kelly 
describes  reveals  the  dynamics  of  this 
struggle.  To  begin  with,  practitioners 
believe  that  asking  adult  learners  to  set 
their  own  goals  makes  sense;  they  are 
adults  and  know  what  they  need  out 
of  a class  or  program.  However,  when 
practitioners  abdicate  their  own  power 
by  not  helping  learners  do  this  often 
unfamiliar  task,  everyone  can  end  up 
frustrated.  In  the  end,  a participatory 
approach  to  goal  setting  means  that  both 
learners  and  practitioners  need  to  have  a 
voice  in  the  process.  Both  parties  need  to 
trust  that  the  other  will  be  honest  and 
explicit  about  their  agenda  and  will 
contribute  their  particular  experience 
and  expertise.  Empowerment  is  not 
something  that  adult  literacy  practitio- 
ners can  give  to  adult  learners;  it  is, 
instead,  both  the  process  and  the  prod- 


uct of  shared  participation  and  critical 
reflection. 

I can  almost  'see'  empowerment  in 
the  pages  of  Adventures  in  Assessment ; 
learners  and  teachers  both  have  a voice 
in  the  assessment  practices  that  practitio- 
ners describe.  In  terms  of  where  to  go 
next,  the  authors  in  Volume  5 point  the 
way.  Practitioners  need  to  invite  learners 
into  the  conversation  about  assessment, 
not  just  into  the  assessment  practices 
themselves.  I look  forward  to  hearing 
more  from  learners  about  what  assess- 
ment practices  are  valuable  to  them  in 
future  issues.  I also  thank  the  practitio- 
ners who  have  contributed  to  Adventures 
in  Assessment  so  far;  they  have  begun  a 
powerful  conversation. 
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In  every  literacy  setting  I've  worked 
in.  I've  wanted  to  change  the  rules. 
The  new  rules  would  be:  this  is  a 
class  where  evaluation  doesn't  hurt  self- 
esteem, where  learners'  knowledge 
about  the  world  gets  counted  and  woven 
into  the  learning  at  hand,  where  the 
work  that  is  done  in  the  class  has  a 
purpose  or  use. 

During  this  past  year  I worked  with 
16  ABE  Math  Team  teachers,  all  intent 
on  changing  the  rules  to  make  our 
classes  look  more  like  a hands-on  math 
lab  of  the  21st  century.  (To  read  more 
about  our  vision  and  the  results  of  the 
shifts  in  math  instruction,  see  The 
Massacusetts  ABE  Math  Standards  Project 
Vol.  1 and  Vol.  2.) 

Much  of  the  time,  adult  learners  will 
politely  go  along  with  whatever  the 
teacher  has  in  mind.  Often  the  learners 
who  come  and  stay,  however,  are  em- 
powered enough  to  ask  that  the  class 
meet  some  felt  need.  How  frustrating  it 
is,  though,  when  learners  request  some- 
thing reminiscent  of  the  traditional 
schooling  I was  working  against.  And  it 
happens  in  every  setting.  The  class 
wants  to  read  orally  round  robin  instead 
of  practicing  silent  reading.  The  students 
in  one-to-one  want  a pure  phonics 
curriculum,  the  math  class  wants  me  to 
correct  their  worksheets. 

Bonnie  Mullinix  and  the  research  we 
did  together  are  responsible  for  the  ideas 
that  have  at  least  temporarily  gotten  me 
out  of  this  pickle  jar.  On  an  abstract 


level,  the  answer  is  continuing  educa- 
tion. We  have  to  educate  students  to  the 
alternatives.  If  they  haven't  heard  of 
dialogue  journals  or  cooperative  learn- 
ing puzzles,  I shouldn't  be  surprised  that 
students  aren't  asking  for  them.  We  have 
to  be  clear  about  our  own  views  and  the 
reasons.  But  sometimes  such  a forthright 
approach  doesn't  work.  I need  to  make 
my  point  concretely  without  getting  on  a 
soapbox,  lecturing,  or  preaching. 

Meeting  Math  Goals 

This  paper  lays  out  a strategy  for 
familarizing  students  with  a broad  view 
of  mathematics,  for  opening  the  dia- 
logue about  which  topics  should  get 
covered  during  math  class,  and  for  using 
assessment  as  an  opportunity  to  build 
expectations  for  a new  or  continuing 
class.  In  this  article  I am  recommending 
a way  to  meet  these  goals.  Listen,  it's 
CHEAP  - not  a lot  of  materials,  not  a lot 
of  time.  Teachers  can  add  this  strategy  to 
a repertoire  of  ways  to  create  situations 
in  which  students  become  familiar  with 
a wider  range  of  instructional  options 
and  are  therefore  more  informed  when 
they  exercise  their  right  to  choose. 

The  strategy  is  simple  and  could  be 
adapted  to  serve  as  an  initial  classroom 
assessment,  as  an  interim  evaluation  to 
get  student  feedback  on  next  steps,  or  as 
an  ongoing  or  final  evaluation  to  com- 
pare students'  ideas  and  comfort  with 
mathematics  with  their  ideas  at  the 
beginning  of  the  program. 
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Topics  on  Cards 

I gave  students  in  pairs  or  groups  of 
three,  a set  of  twelve  cards.  I used  this 
Topic  on  Cards  approach  three  different 
times.  In  one  case  I worked  with  the 
whole  class.  In  the  other  two  cases,  I 
asked  the  teachers  to  choose  two  stu- 
dents they  thought  would  be  interested. 

Each  of  the  twelve  cards  had  a math 
topic  on  it.  (See  Figures  1-3.)  For  example: 

Estimation 

Guessing  about  how  much  something 

mil  be 

Whole  numbers 

(Computation) 

+ - x : 


I simply  said,  'Tm  going  to  give  you 
a bunch  of  cards.  Each  of  these  cards  has 
a math  topic  on  it.  I want  you  to  look 
through  them  and  put  them  in  order. 

Put  the  thing  you  think  is  most  impor- 
tant on  the  top.  Put  what  you  think  is  the 
least  important  on  the  bottom." 

With  basic  level  readers,  I read  the 
topics  with  them.  Usually  the  topics 
didn't  need  elaboration,  although  I 
found  myself  offering  some  examples. 
The  students  needed  little  else  to  work 
on  the  task. 

Some  learners  did  need  help  work- 
in  p-  well  together.  Althoueh  there  were 

o u *-» 

no  stated  rules  about  coming  to  consen- 
sus, I wanted  both  students  in  a pair  to 
participate  actively.  I checked  in  with  the 
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Fractions 

Decimals 
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tations of  math  in 
adult  ed. 
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FIGURE  2 

Assessment  Cards  II 

sed  in  RABEM  Study 

Patterns  and 

Measurement 

Relationships 

Problem  Solving 

Reasoning 

Does  it  make  sense? 

quieter  student,  saying,  "Do  you  agree 
with  that?"  or  "Didn't  you  want  to  keep 
estimation  at  the  top?"  When  one  stu- 
dent seemed  to  have  lost  track  of  the 
purpose,  I reminded  her  "This  is  about 
your  opinion.  What  do  you  think  is  most 
important  for  an  adult  to  know?  There  is 
no  right  answer." 

Once  students  had  finished  and  I had 
recorded  the  outcome,  I asked  them  to 
use  the  same  cards  to  show  me  "which 
topics  you  spend  the  most  time  on  in 
class.  Put  the  ones  you  do  the  most  at  the 
top  and  the  ones  you  don't  do  much  or 
barely  touch  at  the  bottom."  In  both 
tasks,  I said  that  ties  between  two  of  the 
topics  were  allowed.  In  other  words,  two 
cards  could  occupy  the  same  place  in  the 
order  by  being  positioned  side  by  side. 


What  Emerged 

This  activity  opened  three  new 
doors.  A rush  of  new  ideas  flowed  in  to 
mingle  with  the  typical  expectations  of 
math  in  adult  ed.  They  are: 

• Relevance  of  math  to  life  rather  than 
to  the  test 

• New  awareness  of  the  range  and 
breadth  of  math 

• Placement  of  computation  in  perspec- 
tive. 

These  results  are  true  for  all  49 
learners  Bonnie  and  I interviewed.  I've 
chosen  to  focus  here  on  the  nine  learners 
I interviewed. 

Students  cited  relevance  to  life  as 
one  criterion  for  a high  ranking.  Two  of 
the  three  groups  rated  decimals  as  more 
important  than  fractions.  One  student 
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explained  that  "if  you  don't  know  where 
that  decimal  is,  you  can't  tell  about  how 
much  to  expect.  In  a bank  or  a store,  they 
could  give  you  forty  dollars  instead  of 
four  hundred.  You  have  to  know  where 
that  decimal  is."  Without  a secure 
knowledge  of  decimals,  adults  are 
vulnerable  in  the  world  of  money.  The 
third  group  gave  decimals  and  fractions 
equal  importance,  but  none  of  the 
groups  invoked  the  GED  as  their  criteria 
for  importance. 

In  another  instance,  measurement 
was  consistently  above  geometry  for  all 
the  groups.  Though  geometry  figures  on 
the  GED,  measurement  is  a skill  called 
for  in  daily  life.  In  one  class,  a learner 
asked  me  to  explain  what  geometry  is. 


On  reflection,  I realized  this  lack  of 
information  is  indicative  of  a much 
larger  gap.  Many  adults  lack  an  overall 
sense  of  mathematics  as  a discipline/  as 
an  area  of  study.  I can  hear  impatient 
voices  asking,  why  should  someone  who 
needs  to  leam  addition  and  subtraction 
be  burdened  with  explanations  of  geom- 
etry or  calculus?  Isn't  that  jumping  the 
gun? 

No,  it's  not.  Our  students  are  con- 
sumers of  education.  It  is  critical  for 
adults  as  students,  parents,  and  citizens, 
to  have  some  sense  of  the  topics  that  lie 
ahead  of  them,  the  kinds  of  math  their 
children  will  study,  the  disciplines  that 
comprise  scientific  endeavor.  Further- 
more, an  attitude  toward  math  informed 


FIGURE  3 

Assessment  Cards  Used  in  RABEM  Study 


Communication 

Geometry  and 

Symbolic,  graphic,  text 

Spatial  Sense 

How  do  you  read  and  say 
different  math  symbols? 

Understanding  shapes 
and  spaces 

Statistics 

Algebra 

and  Probability 

Collecting  and  organizing 
information,  reading  graphs,  etc. 

By  the  act  of 
choosing  problem 
solving  or  reason- 
ing in  their  top 
five  important 
topics,  they 
commit  them- 
selves to  learning 
concepts  that  will 
help  them  in  those 
areas. 
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by  limited  experience  with  computation 
may  well  have  an  adverse  effect  on  their 
drive  to  learn.  If  they  feel  that  math  will 
only  get  harder  and  be  even  more  te- 
dious than  the  times  tables,  or  more 
complicated  than  adding  fractions  with 
different  denominators,  they'll  pace 
themselves  more  slowly.  Like  people 
anticipating  an  ordeal,  they'll  make  the 
tasteless  thing  they  are  working  on  drag 
on  as  long  as  possible.  Subconsiously 
they  may  hope  that  time  will  run  out 
before  they  have  to  go  on  to  the  next 
thing,  and  in  the  meantime,  at  least  their 
struggle  will  be  familiar. 

I noticed  learners  initially  reach  for 
the  Whole  Numbers  card.  One  student 
asked  "Where's  addition.  You're  going 
to  need  that  even  if  you  aren't  going  to 
know  anything  else."  In  the  course  of 
reviewing  the  other  cards,  however, 
learners  began  to  place  whole  numbers 
in  perspective.  All  three  groups  pulled 
communication  out  and  put  it  on  the 
top.  Without  an  understanding  of  sym- 
bols, you  wouldn't  know  whether  to  add 
two  numbers  or  multiply  them.  Without 
communication,  you  couldn't  explain 
your  answer,  couldn't  explain  that 
you'd  been  given  you  the  wrong  change. 
Without  problem  solving,  "you  can't 
solve  the  problem;  you're  lost."  The 
learners  were  clearly  still  attached  to 
computation,  but  they  were  seeing  it  in 
the  context  of  other  mathematical  skills. 
Since  members  of  the  Math  Team  and 
other  teachers  are  experimenting  with 
the  math  curriculum,  I was  interested  to 
see  that  students  wouldn't  necessarily  be 
wedded  to  tradition.  The  responses  to 
this  task  indicate  that  learners'  expecta- 
tions of  math  class  can  be  quite  flexible. 


Using  this  Strategy 

To  adapt  this  activity  for  initial  or 
ongoing  assessment,  I would  ask: 

Which  topics  would  you  like  to 
spend  most  time  on  in  class?  Put  the 
cards  in  order  from  most  time  to  least 
time. 
or 

Which  topics  do  you  think  are  most 
important  for  us  to  cover  during  the  next 
cycle? 

and 

Which  topics  do  you  feel  most 
comfortable  with?  Put  those  at  the  top, 
the  least  comfortable  at  the  bottom. 

I would  phrase  follow-up  questions 
to  get  at  the  thinking  behind  the  order. 
To  maximize  the  open  lines  of  communi- 
cation about  instruction,  I would  connect 
what  I had  planned  to  do  to  what  the 
student  was  asking  for.  For  example, 

"I  see  you  want  to  learn  fractions.  When 
you  start  class,  we'll  be  doing  measure- 
ment with  whole  numbers.  As  we  get 
into  it,  you'll  be  able  to  begin  fractions 
and  see  how  they  work  using  rulers."  Or 
"You  said  that  problem  solving  is  impor- 
tant We  will  do  that  in  two  ways  during 
math  class.  We  do  problems  together  as 
a group  that  you  have  to  talk  about 
(point  to  communication)  and  you'll 
have  problems  to  take  home  as  well." 

The  Context 

Where  This  Activity  Comes  From 

This  activity  came  about  as  part  of 
a larger  study  from  February  to  May, 
1993. 1 worked  with  Bonnie  Mullinix  of 
World  Education  on  the  Research  into 
Adult  Basic  Education  Mathematics 
(RABEM)  project  sponsored  by  the 
Federal  Department  of  Education.  The 
primary  purpose  of  RABEM  was  to  get 
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a really  good  picture  of  math  instruction 
in  Massachusetts  ABE  programs.  To  do 
that,  program  coordinators  and  teachers 
filled  out  surveys.  Bonnie  and  I inter- 
viewed teachers,  observed  classes  and 
interviewed  students. 

One  aspect  of  the  larger  goal  was  to 
figure  out  where  Massachusetts  adult  ed 
classes  are  in  relation  to  the  Curriculum 
and  Evaluation  Standards  for  School  Math- 
ematics published  by  the  National 
Council  of  Teachers  of  Mathematics. 

The  cards  we  gave  students  came  in  part 
from  the  standards  NCTM  put  forth. 

A few  miner  changes  were  made  based 
on  teacher  interviews  which  preceded 
the  learner  interviews.  For  example. 
Number  Sense  was  too  vague  for  teach- 
ers, so  we  left  it  out  of  the  student 
interview.  A few  other  changes  came 
from  learners  themselves.  When  stu- 
dents from  the  Jackson-Mann  ABE  class 
helped  design  the  learner  interview,  they 
identified  terms  which  needed  reword- 
ing or  clarification. 

The  card  part  of  the  learner  interview 
came  after  we  had  asked  students  ques- 
tions about  their  instructors,  present  and 
past  (how  do  you  think  your  teacher 
feels  about  math?  how  do  you  know?); 
to  define  math  (what  do  you  think  math 
is  anyway?);  and  how  they  liked  math 
(what  they  liked  most  and  least).  While 
we  made  no  effort  to  convert  anyone  to  a 
holistic  view  of  math,  the  types  of  ques- 
tions we  asked  certainly  geared  students 
to  think  about  math  from  many  different 
angles. 


Conclusion 

Now  when  I hear  about  students' 
negative  response  to  an  innovative  math 
class  (like  "This  isn't  math.  When  are  we 
going  to  get  back  to  long  division?"),  I 
think  this  kind  of  exercise  would  really 
help.  By  doing  it,  students  are  reminded 
that  math  is  bigger  than  the  whole 
number  computation  they  are  used  to. 

By  the  act  of  choosing  problem  solving 
or  reasoning  in  their  top  five  important 
topics,  they  commit  themselves  to 
learning  concepts  that  will  help  them  in 
those  areas. 

The  excitement  I felt  at  discovering  a 
successful  new  strategy  to  work  my  way 
out  of  a pickle  was  undoubtedly  intensi- 
fied by  die  use  of  a manipulative,  in  this 
case,  Ae  cards.  I suppose  some  readers 
may  already  be  planning  to  turn  this 
idea  into  a checklist.  In  this  case,  a 
checklist  would  turn  a hands-on  activity 
into  a two-dimensional  task.  It  will  shut 
off  creative  thinking.  It  will  limit  owner- 
ship because  if  the  teacher  holds  it,  she 
will  retain  control  and  if  she  gives  the 
page  to  the  student,  that  act  will  turn  the 
activity  into  a written  task,  like  a test,  a 
medium  which  literacy  students  gener- 
ally find  nerve-wracking.  Cards  are 
more  open-ended.  The  task  gives  stu- 
dents a chance  to  be  creative  in  their 
lay-out  and  to  exhibit  their  organiza- 
tional strategies  in  a way  reading  and 
writing  assessments  don't  usually.  The 
cards  allow  for  pyramids,  diamonds  as 
well  as  stairs  or  a linear  arrangement. 
Because  one  can  talk  while  moving  the 
cards,  they  encourage  an  external  think- 
ing process.  Physically  moving  the  cards 
around  made  the  impact  of  each  decision 
on  the  order  as  a whole  a visible  fact.  In 
one  case,  learners  kept  adding  topics  to 
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the  top  of  the  order  and  we  watched  as 
Algebra,  originally  placed  toward  the 
top  sank  down  further  and  further. 
Manipulatives  and  tactile  learning 
belong  in  assessment. 

As  more  and  more  teachers  imple- 
ment the  NCTM  standards  or  the 
Massachusetts  ABE  Math  Standards  by 
emphasizing  communication  and  prob- 
lem solving  with  calculators  and 
estimation,  some  will  encounter  resis- 
tance from  their  students.  Countering 
student  expectations  in  an  understand- 
ing and  empowering  way  will  be  key  to 
a smooth  transition  to  new  and  fun 
activities  in  the  classroom.  We  need  to 
react  in  ways  that  will  further  the  adop- 


tion of  a new  way  of  doing  math  as  well 
as  to  reassure  students  and  give  them  a 
feeling  of  control  over  what  is  happen- 
ing in  math  class.  Over  the  years  I've 
learned  that  to  have  a real  conversation 
about  what  learners  want,  I need  to 
provide  some  structure  or  some  scaffold- 
ing. Otherwise,  I'll  hear  the  internalized 
messages  from  a lifetime  of  encounters 
with  a traditional  approach  to  schooling. 
It's  not  fair  to  ourselves  as  teachers  or  to 
our  students  as  consumers  of  education 
to  present  choices  without  providing 
learners  with  the  information  they  need 
to  make  reasoned  and  informed  choices. 
Topics  on  cards  are  one  simple  strategy. 
Let's  build  a repertoire. 
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A Response  to  AIA; 

Democracy  Begins  in  Conversation 


Do  I read  Adventures  in 

Assessment ? Yes!  Well,  I try. 
Until  recently,  as  the  director 
of  the  National  Clearinghouse  on  ESL 
Literacy  in  Washington,  D.C.,  I received 
literally  stacks  of  adult-literacy  related 
materials  every  week — journals,  unpub- 
lished documents,  curricula,  textbooks, 
junk  mail.  They  came  from  around  the 
United  States  and  a few  from  as  far 
away  as  England  and  Australia.  I men- 
tion this  because  this  position  gave  me 
the  opportunity  to  recognize  what  a 
unique  iorumAdventures  in  Assessment 
really  is.  I have  found  nothing  else  — 
either  in  the  U.S.  or  overseas  — quite 
like  it.  There  are  a few  journals  and 
newsletters  in  the  field  that  speak  to 
concerns  of  teachers,  but  none  I know 
of  engages  the  writers,  readers  and 
responders  in  the  kind  of  honest,  dia- 
logic process  Adventures  in  Assessment 
does. 

I'd  like  to  use  this  occasion  to  thank 
all  of  you  who  have  written  for  or 
worked  on  Adventures  in  Assessment  for 
the  tremendous  effort  that  keeping  such 
a work  going  must  have  meant.  Your 
experiences  have  become  models  not 
only  for  teachers  looking  for  information 
about  alternative  assessment,  but  also, 

I know,for  many  teachers  out  there 
around  the  country  who  ask  questions 
about  how  to  make  learner-centered  and 
participatory  education  work  for  them 
and  who  feel  lonely  and  isolated  in  their 
own  communities.  Your  articles  allow 


us,  as  readers,  to  feel  like  we  are  truly  in 
the  midst  of  a conversation  about  teach- 
ing and  learning.  For  me  this  was  espe- 
cially true  of  the  articles  that  are  fol- 
lowed up  in  subsequent  issues  with 
reflections  on  how  the  learners  are 
progressing,  which  lessons  worked  or 
didn't  work,  and  what  continues  to  need 
to  be  done.  Janet  Kelly's  articles  about 
being  continually  reminded  of  the 
importance  of  consistent,  frequent, 
informal  communication  with  the 
learner,  as  well  as  her  on-going  reflec- 
tions on  various  assessment  tools  that 
"seemed  perfect  until  put  to  practical 
use"  is  one  example.  Janet  Isserlis'  article 
about  her  student  Rosalie,  and  the 
follow-up  story,  where  she  describes 
how  Rosalie  proudly  took  the  article 
written  about  her  home  to  show  her 
family,  is  another. 

I would  like  to  comment  on  one 
topic,  touched  on  by  John  Comings  in 
Volume  3.  In  his  letter  he  expressed  the 
view  that  assessment  can,  at  times,  be  a 
red  herring,  drawing  attention  away 
from  the  need  for  staff  and  program 
development.  He  implied,  I believe, 
among  other  things,  that  it  would  be 
useful  for  teachers  to  direct  their  atten- 
tion not  only  toward  what  happens 
inside  the  classroom  with  students,  but 
also  toward  what  happens  outside  the 
classroom.  Massachusetts  is  a state 
where  some  of  the  most  innovative  work 
in  staff  and  program  development,  as 
well  as  assessment,  is  taking  place.  Many 
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of  the  most  innovative  curricula  and 
assessment  tools  have  had  their  origins 
there.  Teachers  involved  in  these 
projects,  I believe,  could  do  much  for  the 
field  if  they  were  to  find  ways  to  apply 
the  critical  analysis  process  they  use  in 
the  classroom  outward  to  examine  how 
institutional  supports  and  constraints 
affect  their  work.  While  "changing  the 
system"  in  the  current  social  and  politi- 
cal climate  may  seem  overwhelming  at 
best,  I know  there  are  people — even 
people  in  Washington  offices,  believe 
me — who  would  read  your  observations 
about  your  working  conditions  as 
teachers  with  attention  and  respect.  We 
all  know  the  status  of  the  adult  educa- 
tion workforce  in  the  U.S.  Most  of  us 
struggle  for  excellence  in  the  classroom 
as  part-time  employees  with  no  benefits, 
no  job  security,  few  opportunities  for 
professional  development,  a lack  of 
coordination  among  programs,  and 
unrealistic  funding  guidelines.  The 
teachers  who  write  for  Adventures  in 


Assessment  are  in  a good  position  to  take 
up  the  challenge  by  reflecting  upon  how 
those  conditions  affect  their  work  and 
how  they  might  be  improved. 

I heard  somewhere  that  when  educa- 
tor John  Dewey  was  asked,  toward  the 
end  of  his  life,  what  he  had  learned  in 
all  his  years,  his  answer  was  elegant 
yet  simple.  "I  learned,"  he  said,  "that 
democracy  begins  in  conversation." 
Adventures  in  Assessment  is  special 
because,  in  this  fast-paced  world,  it 
engages  us  as  teachers  and  learners  in 
true  conversations.  We  need  to  cherish 
and  nourish  this  activity  as  a genuine 
way  to  support  one  another  as  teachers, 
to  improve  what  happens  in  the  class- 
room, and  to  find  better  ways  to  include 
learners  in  our  conversations.  But  we 
also  need  to  extend  our  attentions 
outward  to  educate  those  who  hold  the 
power  to  make  decisions  about  funding 
for  literacy  about  the  working  conditions 
that  affect  our  teaching  and  our  lives. 
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Affirmation  for  Pre-Goal  Setting; 


1 would  like  to  thank  Don  Robishaw 
for  his  insights  presented  in  'The 
Case  for  Pre-Goal  Setting"  article  in 
Volume  5 of  Adventures  in  Assessment. 
More  specifically,  I appreciated  the 
reminder  about  not  assuming  our  learn- 
ers will  proceed  and  succeed  through 
schooling  as  we  did.  I also  valued  the 
ipidLCiiiciiL  of  goal-setting  strategies 
within  the  context  of  the  greater  philoso- 
phy of  self-directed  learning.  Seeing  it 
within  its  larger  context  helps  me  to 
reaffirm  its  value  as  an  empowering  tool 
within  my  classrooms. 

I would  like  to  admit  that  this  article 
led  me  to  think  about  goal-setting  as  a 
"middle  class  phenomena"  in  ways  I 
had  not  previously.  As  an  ESL  teacher,  I 
have  had  many  experiences  where  my 
more  "middle-class",  "linear,  future- 
oriented,  individualistic"  approach  has 
not  facilitated  true  communication  and 
understanding  between  me  and  my 
students.  I have  not  looked  at  my  goal- 
setting activities  from  this  perspective, 
and  I valued  the  opportunity  presented 
in  this  article. 

I have  learned  a great  deal  about 
angles  of  learning,  learning  as  circles  and 
waves,  and  assumptions  behind  "knowl- 
edge" in  working  with  multicultural 
students.  For  years  I have  felt  the 
struggles  between  offering  my  students 
a learning  environment  in  the  forms  they 
are  used  to  versus  the  forms  lam  used 
to.  There  has  become  a place  in  me 
where  being  discriminatory  about 


learning  situations  feels  appropriate. 
Teaching  using  a strict  lecture  format 
doesn't  work  for  me,  even  though  it  may 
be  what  my  students  are  accustomed  to. 
Negotiating  both  the  form  and  content  of 
classes  with  the  students  is  always  the 
cutting  edge  of  my  learning  as  a teacher. 

I like  your  suggestion  of  channelling 
students7  former  resist^mce  to  schooling 
into  the  persistence  needed  to  become 
self-directed  learners.  I'd  like  to  know 
more  about  how  to  assist  students  in 
overcoming  their  resistance.  The  pre- 
goal setting  strategy  of  dialogical 
processes  with  their  peers  seems  like  a 
very  helpful  first  step.  I'd  like  to  know 
what  comes  next.  Self-reflective  pro- 
cesses help  all  of  us  as  learners,  and  I 
believe  introducing  them  more  formally 
into  our  teaching  structure  is  very 
helpful. 

"It  is  very  important  that  the  facilita- 
tor has  had  similar  life  and  schooling 
experiences  as  the  learners,  to  develop 
solidarity  with  them." 

Although  I see  this  as  an  ideal  scenario, 
it  seems  to  presume  that  we  cannot 
develop  solidarity  with  learners  if  we  are 
not  from  the  same  background.  I'd  like 
to  strongly  disagree  with  this,  and  offer 
other  areas  where  we  can  form  bonds 
with  our  students.  First  of  all,  the  issues 
of  struggle  are  not  new  to  any  (or  most) 
of  us.  Of  course,  there  is  a range  of  levels 
of  struggle  — yet  I believe  it  is  a com- 
mon thread  of  the  human  experience 
, that  we  can  easily  draw  upon  to  form 
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webs  of  connection  with  our  students. 
Furthermore,  I play  many  similar  life- 
roles  as  my  students  do:  mother, 
daughter,  bread-winner,  partner,  etc., 
each  of  them  offering  the  food  for  build- 
ing connections.  But  far  more  important 
than  these  is  acknowledging  the  learning 
environment  as  a place  where  collabora- 
tion presides,  where  "professors  and 
students  actively  and  mutually  engage 
in  the  learning  process.  Together,  they 
define  and  create  a body  of  knowledge  that 
informs  and  transforms  our  ux>rM(N.E.A., 
p.8).  This  is  where  we  primarily  develop 
solidarity  with  the  learners.  We  are 
paired  in  a co-creative  process,  as  both 
learners  and  teachers.  When  we  bring 
this  level  of  equality  and  respect  to  our 
classrooms,  acknowledging  our  role  as 
life-time  learners,  we  create  an  environ- 


ment that  supports  us  all  in  taking  risks, 
advocating  for  our  needs  and  being  the 
"experts".  In  my  workplace,  we  use  the 
term  "communities  of  scholars"  to 
denote  this  philosophical  and  pedagogi- 
cal belief/structure.  It  is  my  experience 
that  when  allowed  to  see  ourselves  as  an 
integral  piece  within  the  structure  of  a 
whole  unit  (class),  we  synergize  in 
creating  a dynamic  whole  which  is 
defined  by  our  needs  and  personal 
differences,  and  whose  goals  are  to  work 
together  to  serve  our  collective  and 
individual  needs.  Thus,  far  more  impor- 
tant than  having  the  same  background 
as  our  students,  is  having  the  same 
leverage  and  power  in  our  present 
learning  situations.  This  truly  facilitates 
respect  and  solidarity. 
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Learning  from  Experience 

Fiona  Minnow  to  Oveiachievei 


(The  following  is  an  account  of  my  personal 
experiences  with  being  evaluated  and 
assessed  as  a learner.  My  road  to  participa- 
tory assessment  is  due  in  part  to  these  early 
painful  experiences.  These  visceral  experi- 
ences have  contributed  substantially  to  my 
opposition  and  resistance  to  any  kind  of 
standardized  testing.  In  addition  reflecting 

OulK  OH  tncdc  cXf/cTicTtCcS  has  TTludC  77 1C  VCTy 

empathetic  to  other  learners  who  have 
experienced  evaluation  as  a less  than  positive 
experience.) 

T here  are  four  events  in  my  life 
that  have  led  me  to  learner- 
centered  approaches  to  assess- 
ment. The  first  event  occurred  while  I 
was  in  2nd  grade.  I was  a "minnow". 
While  all  my  girlfriends  were  swimming 
through  the  blue,  green,  yellow,  and  red 
versions  of  the  endless  and  exciting  tales 
of  Sally,  Dick,  and  Jane,  I was  stuck  on 
"See  Spot  Rim".  I was  stuck  in  the  back 
of  the  room  with  the  rowdy  boys  (my 
friends  on  the  baseball  diamond)  and  the 
other  "slow  girls".  The  only  thing  that 
motivated  me  to  "grow  up"  and  become 
a "goldfish"  and  eventually  a "shark" 
was  the  glimmer  of  hope  that  one  day  I 
would  be  able  to  read  a chapter  book,  a 
hard  cover  book  that  maybe  had  a story 
worth  knowing.  This  was  event  #1.  You 
can  tell  from  my  tone  that  I've  been 
scarred  for  life — an  adult  child  of 
tracking  and  basal  readers. 

My  second  event  happened  the 
summer  of  5th  grade.  Somehow  I had  . 


learned  to  read  by  the  time  I got  to  5th 
grade  but  math  was  still  a problem  for 
me.  My  parents  had  tried  everything 
from  summer  school  intensives  to  the 
"teaching  machine".  Knowing  math 
concepts  or  understanding  how  math 
could  be  used  in  one's  daily  life  hadn't 
happened  yet. 

So  my  parents,  in  a desperate  attempt 
to  do  their  jobs  as  good  parents,  bought 
me  a "teaching  machine",  a small  blue 
box  that  you  put  paper  into  then  scrolled 
up  endless  math  problems  to  solve  over 
and  over  again  until  you  got  it  right.  It 
was  the  precursor  to  doing  drills  on  the 
computer. 

I was  supposed  to  do  a certain  num- 
ber of  "units"  a day  before  going  out  to 
play  or  swim.  After  about  the  third  day 
sitting  inside  a hot  stuffy  room  by  myself 
faying  to  solve  algebra  problems  that  I 
more  often  than  not  got  wrong,  I discov- 
ered that  when  I put  the  paper  into  the 
machine  I could  see  all  the  answers.  The 
solution  to  my  dilemma  was  simple;  I 
would  simply  copy  the  correct  answers 
(but  not  all  the  answers  because  then  I 
would  get  caught).  I figured  out  a certain 
percent  I should  get  right  and  then  I 
would  increase  it  gradually  so  no  one 
would  suspect.  The  system  worked 
great.  I did  my  assigned  summer  work 
in  less  than  half  the  time  it  normally 
would  have  taken  me  and  indirectly 
learned  something  about  percents. 

I also  learned  something  about  math 
and  learning  too.  I learned  I could  beat 
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I was  now  being 
punished  for 
going  beyond  my 
assigned  level  of 
skill  and 

experience  as  well. 
I was  condemned 
and  furthermore 
I could  be  thrown 
out  of  school  for 
poetic  inspiration. 
This  was  worse 
than  just  being 
labeled  "slow". 

I was  being 
stigmatized  as  an 
"overachiever" — 
a person  who 
achieved  more 
than  they  were 
supposed  to. 
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the  system  if  I really  set  my  mind  to  it 
and  that  getting  the  right  answer  didn't 
feel  like  learning  whether  I cheated  or 
worked  on  the  problem.  This  second 
revelation  came  about  when  I decided  I 
would  not  just  copy  the  answer  but 
figure  out  why  it  was  the  right  answer 
and  then  work  myself  backwards  into 
figuring  out  how  this  could  be.  As  a 
result  math  became  interesting  and 
challenging  and  I think  I actually  learned 
something  in  the  process. 

My  third  event  happened  in  10th 
grade  in  my  English  class.  We  were 
assigned  an  essay  to  do  over  the  week- 
end. I wrote  about  death  and  dying  a la 
Barry  McQuire's  "The  Eve  of  Destruc- 
tion", George  Harrison  and  John  Donne. 
An  odd  combination  of  influences  I 
agree,  but  this  was  the  1960s  andT  was 
an  anti-war  protester  and  life  was  like 
that  back  then.  I wrote  this  essay  and, 
what  can  I say,  I was  divinely  inspired 
when  I wrote  it.  The  words  just  poured 
out  of  me  and  I trusted  them.  I turned 
my  paper  in  with  a feeling  of  having 
done  a great  job  for  once.  I got  it  back  a 
few  days  later  with  a big  red  letter  F on 
it  and  a "See  me"  scrawled  next  to  it  I 
was  in  a state  of  shock.  How  could  I 
have  been  so  wrong  about  my  accom- 
plishments? 

The  teacher  (also  our  football  coach) 
said  my  essay  was  "unbelievably  well 
written  and  mature  and  hence  obviously 
not  mine;  I must  have  plagiarized  and 
plagiarism  was  cause  for  expulsion.  I 


didn't  know  what  to  say.  I was  now 
being  punished  for  going  beyond  my 
assigned  level  of  skill  and  experience  as 
well.  I was  condemned  and  furthermore 
I could  be  thrown  out  of  school  for 
poetic  inspiration.  This  was  worse  than 
just  being  labeled  "slow".  I was  being 
stigmatized  as  an  "overachiever" — a 
person  who  achieved  more  than  they 
were  supposed  to. 

My  final  event  happened  during  my 
senior  year  of  high  school.  I had  applied 
to  a college  which  will  remain  nameless, 
though  I will  never  forget  that  bitter 
autumn  day  in  the  admission  officer's 
book-cluttered  office.  I had  just  finished 
a tour  of  the  campus  and  was  now 
chatting  with  the  admissions  officer 
about  Thomas  Aquinas  and  how  I 
wanted  to  continue  my  education  in 
philosophy.  Yes,  I had  read  already  most 
of  the  Great  Books.  In  the  middle  of  our 
conversation  he  said  he  hadn't  received 
my  board  scores  yet  but  as  soon  as  they 
came  in  he  would  let  me  know  about  the 
college's  decision.  He  called  his  assistant 
who  came  in  with  them  in  her  hand.  I 
watched  his  face  go  from  concerned  to 
sad  as  he  shook  his  head.  Finally  he  said 
that  though  I had  "wisdom,"  I didn't 
have  the  academic  qualifications  to  go  to 
this  school.  Wisdom  but  low  test  scores. 
This  was  my  final  run  in  with  the  system 
and  its  need  to  evaluate  me  not  accord- 
ing to  what  I knew  but  how  I measured 
against  the  "norm". 
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Book  Review 

Portfolios  In  the  Writing  Classroom 


Yancy,  K.  B.  (Ed.)  (1992).  Portfolios 
in  the  Writing  Classroom.  Urban  a,  IL: 
National  Council  of  Teachers. 

Alternative  approaches  to 

Assessment  that  enables  adult 
L literacy  students  to  evaluate 
their  own  experiences  and  progress 
come  to  us  in  many  forms  today.  These 
approaches  help  students  view  their 
own  learning  process  in  writing.  They 
also  help  teachers  identify  the  strategies 
students  use  and  how  these  strategies 
change  as  they  progress  as  learners.  The 
use  of  portfolios  as  an  assessment  tool  is 
one  approach  that  has  become  quite 
popular  in  our  field  recently. 

Many  adult  literacy /adult  basic 
education  (AL/ ABE)  practitioners  are 
using  portfolios  in  their  writing  class- 
rooms. Portfolios  in  the  Writing  Classroom 
contains  a range  of  essays  on  that  very 
subject. 

The  book  begins  with  an  essay  that 
discusses  the  relationship  among  portfo- 
lio assessment  summative  assessment 
and  formative  assessment.  It  also  dis- 
cusses what  might  be  called  "reflective 
evaluation/'  a form  of  self  evaluation 
considered  very  important  to  the  writing 
process.  The  use  of  portfolio  assessment 
in  the  writing  class  relies  heavily  on  a 
self  reflective  process. 

The  article  also  deals  with  the  chal- 
lenges that  the  portfolio  movement  in 
the  United  States  feces: 


1)  Weakening  of  effect  through 
careless  imitation  (fear  of  the  "band- 
wagon" effect,  as  happens  in  so  many 
other  areas  of  innovations). 

2)  The  failure  of  research  to  validate 
this  pedagogy.  (Many  still  believe  that 
"it"  has  to  be  measurable  and  countable 
to  be  valid  and  worth  doing.) 

3)  The  co-optation  by  large  scal6 
external  testing  programs.  (There  may 
be  incongruities  between  large  scale 
portfolio  testing  and  writing  portfolios 
used  in  the  classroom.) 

As  in  the  introductory  article,  the 
remaining  authors  are  not  from  an  AL/ 
ABE  background,  but  mostly  from  fields 
related  to  formal  schooling.  Still,  some  of 
these  discussions  will  be  of  interest  to 
the  AL/ABE  professional. 

The  remaining  articles  are  described 
as  follows: 

"Collectively,  the  chapters  reflect  a 
movement  from  the  self  initiated  use  of 
portfolios,  as  narrated  in  Sue  Ellen 
Gold's  chapter,  and  from  the  individual 
struggling  to  make  sense  out  of  a general 
"assignment"  to  introduce  portfolios,  as 
described  by  James  Newkirk,  towards 
the  use  of  portfolios  taken  up  by  teach- 
ers working  together  in  community. 
Catherine  D'Aoust's  teachers  are  still 
working  individually,  but  support  each 
other  in  a university  seminar  on  Teach- 
ers as  Researchers'.  Sandra  Murphy  and 
Mary  Ann  Smith  describe  a middle 
school  faculty  cooperating  with  outside 
researchers  to  learn  how  to  derive 
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It  also  discusses 
what  might  be 
called  " reflective 
evaluation,"  a 
form  of  self  evalu- 
ation considered 
very  important  to 
the  writing  pro- 
cess. The  use  of 
portfolio  assess- 
ment in  the 
writing  class 
relies  heavily  on  a 
self  reflective 
process. 


insights  from  a shared  portfolio  project 
— insights  about  students  and  about 
how  portfolio  projects  work.  Roberta 
Camp's  portfolio  project  grew  out  of 
cooperation  between  theorists,  educa- 
tional testers,  adminis-trators,  and 
teachers  of  the  performing  arts,  for 
whom  portfolio  took  on  a special  func- 
tion as  instruments  for  student  growth 
allowing  assessment  of  the  learning 
processes  as  well  as  the  products. . . 
David  Kneeshaw  discusses  portfolio 
from  an  even  larger  perspective  in  his 
description  of  the  Ontario  "Writing 
Folder"  project,  intended  to  allow 
evaluation  and  record  keeping  as  a 
student  moves  across  grade  levels,  but 
designed  as  well  to  encourage  much  of 
the  same  sense  of  discovery  by  teachers 
and  students  that  characterizes  the 
individual  accounts...  Irwin  Weiser  tells 
the  last  story,  of  a considered  decision  to 
introduce  portfolios  into  the  basic 
writing  program  at  Purdue  University, 
primarily  as  a way  to  defer  summative 
grading"  (pp.  13-14). 

The  book's  editor  concludes  with  a 
short  essay  on  the  importance  of  self 


reflection  and  portfolios  in  the  wiring 
classroom.  She  also  leaves  us  with  many 
unanswered  questions  that  can  be 
perhaps  best  answered  by  the  individual 
teacher  and  his  or  her  individual  stu- 
dents. 

There  axe  a growing  number  of 
articles,  but  only  a few  books  on  the 
subject  of  portfolios,  either  as  an  assess- 
ment tool  for  reading  and  writing  or  an 
approach  to  helping  students  improve 
their  writing.  Kathleen  Blake  Yancy  has 
put  together  this  collection  of  essays  on 
the  use  of  it  as  an  evaluation  tool  and  on 
the  "pedagogy  of  portfolios." 

I especially  recommend  this  book  to 
practitioners  in  our  field  who,  because  of 
the  focus  of  Adventures  in  Assessment, 
and  the  recent  "creative  wave"  of  assess- 
ment tools  by  many  Massachu-sett's 
AL/ABE  practitioners  that  we  may 
primarily  think  of  portfolios  as  an  inno- 
vative approach  to  assessment.  In  reality, 
though,  it  is  more  important  to  think  of 
portfolios  as  an  excellent  self  reflective 
writing  tool  and  as  an  important  peda- 
gogical approach  to  the  teaching  of 
writing. 
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Mission  Statement  from  the  Transformers 


The  Transformers  is  a newly 

formed  group  of  adult  literacy 
practitioners  in  Massachusetts. 
Many  of  its  members  come  from  compo- 
nent #3  of  the  Department  of 
Education's  GOALS  project  which  had 
as  its  mission  the  goal  of  looking  at 
alternative  assessment  or  leamer-cen- 

. a £!*u  1 a 
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The  goal  of  this  team  is  to  create  and 
support  systemic  change  from  hierarchi- 
cal models  of  education  and  social 
organization  to  egalitarian  models, 
where  learners  and  teachers  are  partners 
in  education. 

We  will  accomplish  this  by  maintain- 
ing a dialogue  within  the  field  of  adult 
education  with  a focus  on  participatory 
field-based  assessment.  We  are  dedi- 
cated to  concentrating  and  coordinating 
our  efforts  towards  broadening  the 
understanding  of  adult  educators, 
learners,  administrators,  and  funders 
about  participatory  principles  which  are 
the  basis  for  meaningful  assessment  and 
social  change. 

We  are  committed  to  developing  new 
ways  of  looking  at  our  programs,  our- 
selves, and  our  communities  in  order  to 
support  mindful  action  and  access  to 
power.  Our  vision  is  to  continue  to  build 
a network  of  interested  people  who  will 
work  together  on  projects  that  lead  to 
this  kind  of  change  at  the  program, 
community,  state,  and  national  policy 
level 


PRINCIPLES  OF  PARTICIPATORY 

ASSESSMENT 

1.  It  must  be  program-based  and  learner- 
centered. 

2.  It  should  help  the  learners  achieve 
their  goals. 

3.  It  must  build  on  learner  strengths,  not 
deficits. 

4.  It  must  be  part  of  the  learning  experi- 
ence. 

5.  It  should  not  be  a single  procedure  but 
a variety  of  procedures. 

6.  It  should  provide  feedback  that  will 
lead  to  better  instruction. 


STRATEGIES 

• Collaborate  with  other  groups  and 
individuals  with  whom  we  share 
common  goals  such  as  SABES,  New 
England  literacy  Resource  Center, 
and  community  based  organizations. 

• Continue  to  support  the  development 
of  field-based  assessment  practices 
through  the  Partnership  Project.  The 
Partnership  Project  is  a mentoring 
project  funded  by  DOE  which  allows 
practitioners  who  are  interested  in 
learning  about  participatory  assess- 
ment to  become  partners  with  pro- 
gram staff  who  have  been  developing 
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and  using  participatory  procedures 
and  tools. 

• Disseminate  the  toolkit  to  practitio- 
ners throughout  the  state  and  the  New 
England  Region. 

• Create  and  coordinate  a pool  of 
trainers  in  participatory  assessment  to 
provide  training  and  technical  assis- 
tance within  our  state.  New  England 


and  the  country. 

• Maintain  our  connection  and  commit- 
ment to  the  journal  Adventures  in 
Assessment 

• Interpret  and  clarify  the  Massachu- 
setts Quality  Indicators  with  a focus 
on  how  portfolio  assessment  fits  in  to 
the  state's  policy  on  assessment. 
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SURVEY 

Please  fill  out  and  return  to  Loren  McGrail  at  SABES 

(address  on  inside  back  cover) 

This  survey  is  to  determine  the  effect  that  Adventures  in  Assessment  Learner  Centered  Approaches  to  Assessment 
and  Evaluation  has  had  on  the  field.  Does  the  journal  either  change  or  validate  people's  practices  and  is  it  a good 
staff  development  tool?  We  define  staff  development  as  the  process  of  reinforcing,  maintaining  and  building 
effective  teaching  practices.  Ultimately,  staff  development  acts  as  an  antidote  to  burnout. 

1 . Have  you  read  Adventures  in  Assessment ? □ Yes  □ No 

If  you  have,  which  volume(s)?  □Volume  1 (Yellow)  □ Volume  2 (Light  Green) 

□ Volume  3 (Purple)  □ Volume  4 (Dark  Green) 

2.  Do  you  find  the  layout/design  of  Adventures  in  Assessment  accessible  or  easy  to  understand?^  Yes  □ No 
Comments: 


What  criteria  do  you  use  when  deciding  which  articles  to  read? 

read  the  introductions?  □ Yes  □ No 
read  Voices  from  the  Field?  □ Yes  □ No 
read  Getting  Started?  □ Yes  □ No 
read  Ongoing?  □ Yes  □ No 
read  What  Counts?  □ Yes  □ No 


□ Titles  or  articles 

Do 

□ Author 

Do 

□ Interests 

Do 

□ Other  (Specify) 

Do 

Comments: 

Do 

Has  Adventures  in  Assessment  affected  your  thinking  or  beliefs? 
Comments: 


□ Yes  ONo 
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5.  Hos  Adventures  in  Assessment  affected  your  practices  (such  as  sparked  a new  tool  orthrown  out  all  of 
them)?  □ Yes  □ No 
Comments: 


6.  Have  you  changed  the  your  assessment  practices  in  your  classroom  and/or  program?  □ Yes  □ No 

7.  Do  you  have  ways  in  which  you  include  learners  in  your  assessment  practices?  □ Yes  □ No  What  are  they? 

'? 

8a.  Have  you  ever  used  any  ideas  or  tools  from  an  article?  □ Yes  □ No  Which  one? 

8b.  Please  comment  on  how  you  used  it.  Did  you  change  it?  Did  you  adapt  it? 


8c.  May  we  publish  these  comments  in  the  "Letter  to  the  Editor"  section  of  the  journal?  □ Yes  □ No 
9.  Would  you  be  interested  in  writing  for  the  journal?  □ Yes  □ No 

If  yes,  please  fill  in  the  information  below  or  call  Loren  McGrail  at  World  Education,  (617)  482-9485. 

Name: 


Address: 

Phone: 
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Funded  by  the  Massachusetts  Department  of  Education 


S A B E S 

System  for  Adult  Basic  Education  Sep  sen 


SABES  is  the  System  for  Adult  Basic  Education  Support , a comprehensive  training  and  technical 
assistance  initiative  for  adult  literacy  educators  and  programs.  Its  goal  is  to  strengthen  and  enhance 
literacy  services  and  thus  to  enable  adult  learners  to  attain  literacy  skills.  SABES  accomplishes  this 
goal  through  staff  and  program  development  workshops ; consultation , mini-courses,  mentoring  and 
peer  coaching,  and  other  training  activities  provided  by  five  Regional  Support  Centers  located  at 
community  colleges  throughout  Massachusetts. 

SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect f evaluate , and  disseminate  ABE 
materials,  curricula,  methodologies,  and  program  models.  Each  of  the  five  SABES  Regional  Support 
Centers  similarly  offers  program  support  and  a lending  library.  SABES  maintains  an  Adult  Literacy 
Hotline,  a statewide  referral  service  which  responds  to  calls  from  new  learners  and  volunteers.  The 
Hotline  number  is  1-800-447-8844. 

The  SABES  Central  Resource  Center,  located  at  World  Education,  publishes  a statewide  quarterly 
newsletter,  “Bright  Ideas . ” and  journals  on  topics  of  interest  to  adult  literacy r professionals,  such  as 
this  volume  of  “Adventures  in  Assessment.  ” 

The  first  three  volumes  of  “Adventures  in  Assessment”  present  a comprehensive  view  of  the  state  of 
practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners . Volume  1,  Getting 
Started,  included  start-up  and  intake  activities.  Volume  2,  Ongoing,  shares  tools  for  ongoing 
assessment  as  part  of  the  learning  process;  Volume  3,  Looking  Back,  Starting  Again,  focused  on  tools 
and  procedures  used  at  the  end  of  a cycle  or  term,  including  self  class,  and  group  evaluation  by  both 
teachers  and  learners.  Volume  4 covered  a range  of  interests.  Volume  5,  Tale  of  the  Tools  was 
dedicated  to  reflecting  on  component  3 tools  of  alternative  assessment.  Volume  6,  Responding  to  the 
Dream  Conference,  way  dedicated  to  responses  to  Volumes  1-5.  Volume  7,  The  Partnership  Project, 
highlighted  writings  from  a mentoring  project  for  practitioners  interested  in  learning  about 
participatory  assessment.  Volume  8 covered  a range  of  topics  from  education  reform  to  learner 
involvement  in  assessment.  Volume  9 looked  at  assessment  in  a volunteer  program,  native  language 
literacy,  ESOL  programs,  and  evaluations  in  workplace  education  programs. 

We  would  like  to  see  your  contribution,  especially  if  you  are  doing  native  language  literacy  or  low- 
level  ESOL  literacy.  Contact  editor,  Alison  Simmons,  to  discuss  your  submission. 

Permission  is  granted  to  reproduce  portions  of  this  journal.  We  request  appropriate  credit  be  given 
to  “Adventures  in  Assessment"  and  to  the  author. 


“Adventures  in  Assessment ” is  free  to  DOE-funded  Massachusetts  programs ; out-of-state  requests 
will  be  charged  a nominal  fee,  at  cost.  Please  write  to,  or  call: 


Alison  Simmons 
SABES  Central  Resource  Center 
World  Education 
44  Farnsworth  Street 
Boston,  MA  02210-1211 
617-482-9485 
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Working  Together/  Sharing  Ideas 


The  seventh  volume  of 

Adventures  in  Assessment 
highlights  writings  from  The 
Partnership  Project,  a mentoring  project 
funded  by  the  Massachusetts  Depart- 
ment of  Education  which  allowed  prac- 
titioners interested  in  learning  about 
participatory  assessment  to  become 
partners  with  other  practitioners  who 
have  been  developing  and  using 
participatory  procedures  and  tools.  As 
you  work  your  way  through  the  first 
section  of  this  volume,  you'll  follow  the 
route  of  12  practitioners  who  chose 
mentoring  as  a research  vehicle  to 
explore  the  landscape  of  alternative 
assessment  As  a reader  you  can  become 
involved  by  looking  at  the  articles  as  a 
panorama  of  experiences  that  you  can 
add  to  or  subtract  from,  given  your 
experiences  and  your  program  needs. 


by 


Alison  Simmons 


Editor 

S ABES  Central 
Resource  Center/ 
World  Education 


THI  PARTNERSHIP  WRITERS 

The  writers  involved  in  The 
Partnership  Project  highlight  the  impor- 
tance of  having  the  resources  and  sup- 
port needed  to  come  together  as  peers 
around  a common  theme.  As  a group 
these  articles  raise  a lot  of  questions 
about  assessment  and  mentoring  as 
another  vehicle  for  dissemination  of 
alternative  assessment.  Most  of  the 
writers  here  focus  their  inquiry  and 
development  not  only  in  assessing 
learners  skills  in  a particular  content  area 
but  they  also  recognize  the  importance 
of  looking  deeper  into  the  other  factors 
that  make  learners  teachable.  Rudee 


Atlas  and  Dan  Wilson  respond  to  the 
fears,  anxieties,  and  needs  of  students  at 
the  very  beginning  of  their  learning  who 
have  little  or  no  education  in  their  own 
countries.  Deirdre  McLaughlin,  Marti 
Tassi- Richardson,  Loretta  Pardi  and 
Estelle  Williams  raise  questions  about 
learners  who  come  to  class  with  barriers 
and  issues  that  prevent  them  from  par- 
ticipating in  a traditional  classroom 
setting  as  well  as  setting  and  achieving 
their  goals.  These  practitioners  deve- 
loped tools  that  assess  particular  skill 
areas  (Le.  GED  content  areas)  but  also  in 
the  process  raise  the  self  esteem  of  their 
learners.  It  is  assumed  that  by  raising  the 
self  esteem  of  the  learners  there  will  be 
an  increase  in  motivation  for  learners  to 
continue  their  education  and  see  the 
attainment  of  their  goals  as  a viable 
option. 

With  Lesly  Desire  and  Henry  Joseph 
we  catch  a glimpse  of  the  struggle  they 
faced  when  trying  to  sort  through  the 
assessment  materials  handed  to  them  as 
part  of  The  Partnership  Project  In  their 
vision  of  alternative  assessment  they 
wonder  if  there  is  an  ending  to  this 
inquiry  as  Lesly  states  “It  (assessment)  is 
ever  changing  and  never  finished."  We 
hope  to  get  an  update  on  their  findings 
in  a future  issue  of  Adventures. 

Barbara  Krol-Sinclair  and  Pauline 
O'Leary  work  on  developing  a tool  that 
would  encourage  parents  to  plan  and 
assess  their  literacy  activities  with  their 
children. 
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As  a partnership,  Carolyn  Gear,  co- 
coordinator of  the  project,  and  Widi 
Sumaryano,  mentee,  discuss  the  impact 
this  project  has  had  on  Widi's  learners 
as  well  as  on  his  growth  as  a teacher.  In 
another  type  of  self  reflection  Caroline 
talks  about  the  richness  of  being  a 
mentor  and  the  key  factors  that  make  a 
mentoring  relationship  work. 

To  give  us  an  overview  of  the  project 
and  its  goals  Paul  Trunnel,  co-  coordina- 
tor, talks  about  the  process  and  the  why 
of  the  Partnership  Project.  He  also 
reflects  on  the  process  and  its  impact  on 
dissemination  of  alternative  assessment 
in  Massachusetts. 

With  Don  Robishaw/s  article  we  walk 
away  from  The  Partnership  Project  but 
not  some  of  the  ideas  and  questions  that 
the  group  has  raised.  Don  takes  the 
questions  of  self  esteem  and  goal  setting 
a bit  further  by  saying  that  some  stu- 
dents need  a unlearning  process  before 
they  can  actively  participate  in  these 
activities  in  a meaningful  way  and 
become  self-directed  learners.  He  states 
that  students  may  not  be  ready  for  the 
participatory  processes  that  require 
students  to  define  their  goals  and 
evaluate  themselves  towards  attainment 
of  these  goals.  In  his  article  he  looks  at 
an  intervention  program  that  he  has 
developed  that  addresses  the  question  of 
self  direction  and  student  preparation 
for  goal  setting  and  the  participatory 
classroom. 

In  her  response  to  Judy  Hofer  and  Pat 
Larson's  article  (see  Responding  to  the 
Dream  Conference,  ALA,  Spring  1994), 
Janet  Isserlis  takes  us  on  a questioning 
journey  into  literacy  practices  and 
community  development  by  forcing  us 
to  look  at  how  our  roles  as  practitioners 


are  defined  in  terms  of  community 
development.  In  the  process  of  asking  us 
to  think  critically  about  our  roles,  she 
describes  the  program  she  is  working 
with  in  Vancouver  and  how  she  is 
defining  her  role. 

Kenneth  Tamar  kin  and  Susan 
Barnard  in  "The  Right  Answer"  look  at 
open-ended  questions  as  an  alternative 
process  for  the  ADP  math  assessment. 
They  propose  that  testing  for  critical 
thinking  skills  in  addition  to  computa- 
tional skills  will  give  us  an  understand- 
ing of  the  learners'  thought  processes 
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article  documents  the  process  and  results 
of  their  research  and  examples  of  the 
questions  themselves. 

Debbie  Tuler  discusses  the  revision 
of  an  assessment  tool  in  a workplace 
education  program  in  Newton,  MA.  The 
need  for  revising  an  existing  tool  was  a 
common  concern  between  all  stake- 
holders (learners,  managers,  supervisors 
and  teachers)  that  it  was  not  giving 
them  the  information  needed  to  make 
informed  decisions  about  attending 
classes,  informing  teaching,  and  placing 
learners  in  the  appropriate  classes. 
Debbie  also  talks  about  the  process  of 
revising  the  initial  assessment  and  the 
challenge  of  meeting  the  needs  of  all 
stakeholders. 

In  learning  from  experience,  Elizabeth 
Santiago  talks  about  her  experience  as  a 
GED  recipient  and  raises  the  question  of 
equivalency  and  the  stigma  that  goes 
along  with  the  Alternative  Diploma. 

A poem  about  assessment  is  included 
in  this  volume  from  Diane  Pecarora  in 
Minnesota.  It  is  a first  in  Adventures  in 
Assessment  and  I hope  it  sets  a precedent 
for  those  who  would  rather  poetry,  not 
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prose.  We  will  continue  to  encourage 
alternative  forms  of  expression  as  long 
as  it  fits  into  the  printed  page. 

Lenore  Balliro  reviews  Dimensions  of 
Change:  An  Authentic  Assessment 
Guidebook  by  Melody  Schneider  and 
Mallory  Clarke  from  Seattle,  Washing- 
ton. This  guide  book  is  the  documenta- 
tion of  a special  assessment  project 
entitled  "Integrated  Assessment:  Being 
Accountable  to  Teachers  and  Students" 
where  the  authors  worked  with  teachers 
and  learners  around  assessment  in 
Washington  State. 

Adventures  in  Assessment  has  a new 
Editor  Alison  Simmons  from  SABES 
Central  Resource  Center  at  World 
Education.  Alison  has  a background  in 
ESL  and  is  a literacy  specialist  with 
SABES.  Rick  Schwartz  will  continue  to 
be  the  Assistant  Editor  for  the  journal 
For  future  Adventures  in  Assessment,  we 
will  maintain  the  scope  of  the  journal. 


which  is  practitioner  based,  and  see  how 
we  can  include  the  voice  of  the  learner  in 
the  dialogue.  We  will  also  be  publishing 
Adventures  in  Assessment  once  a year 
each  Fall  to  make  way  for  a new  journal 
through  SABES  on  program  and  staff 
development. 

We  still  encourage  people  to  write  for 
our  various  components  of  getting 
started,  on-going,  looking  back,  what 
counts,  letters  from  the  field,  learning 
from  experience  and  publications 
review.  Although  Volume  8 will 
highlight  some  articles  from  the 
workplace,  we  encourage  people  to 
respond  to  articles  in  this  issue  or  share 
with  us  your  altemative/authentic 
practices. 

(signature  here) 

Alison  Simmons 
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The  Mentoring  Project  has  its 
roots  in  a statewide  project  on 
assessment  standards  initiated  in 
the  summer  of  1992  by  the  Massachu- 
setts Department  of  Education's  (DOE) 
Bureau  of  Adult  Education.  This  project, 
dubbed  Greater  Opportunities  for  Adult 
learner  Success,  or  GOALS,  included 

pdfuupflTtiS  made  Up  Gi  plaCuuDnUS 

from  adult  education,  teachers,  counse- 
lors, administrators,  and  even  students. 
Each  component  focused  on  a different 
aspect  of  assessment  and  examined  its 
viability  as  part  of  a statewide  assess- 
ment system.  While  different  compo- 
nents studied  computer  tracking, 
standardized  examinations,  and  student 
evaluations  of  the  programs.  Component 
Three  focused  on  alternative  assessment 
Comprised  of  practitioners  from  all 
over  Massachusetts,  Component  Three 
was  certainly  not  the  statewide  initiative 
of  alternative  assessment  Many  of  the 
participants  had  already  been  develop- 
ing and  sharing  different  methods  in 
their  programs  for  years.  Yet  it  was  the 
first  occasion  that  alternative  assessment 
was  finally  recognized  by  the  DOE  as  a 
potential  alternative  to  more  tradition- 
ally accepted  methods  (e.g.  standardized 
testing  or  competency-based  curricula). 

It  was  also  the  first  time  that  alternative 
assessment  practitioners  had  assembled 
on  a regular  basis  for  the  purpose  of 
discussing  their  work. 

While  the  other  components  were 
developing  their  projects  with  predeter- 


mined products.  Component  Three 
participants  were  given  a blank  canvas. 
Our  sole  requirement  was  to  investigate 
how  alternative  assessment  could  be 
used  as  part  of  a statewide  assessment 
system.  In  our  discussions,  it  became 
obvious  that  we  all  had  different  ap- 
proaches and  that  these  differences  were 
products  of  our  different  learners  rlsss- 
rooms,  programs,  and  communities,  as 
well  as  our  different  strengths  at  teach- 
ers. As  such,  there  was  clearly  no  one 
form  or  approach  that  was  better  than 
another. 

We  knew  that  our  "product"  as 
Component  Three  wouldn't  be  a stan- 
dardized tool  or  activity  that  could  be 
used  by  all  programs.  Instead,  we 
wanted  something  that  would  communi- 
cate the  importance  of  differences 
among  learners,  teachers,  classes,  and 
programs.  What  better  way  than  to 
combine  all  of  our  different  methods  and 
materials  and  make  those  available  to 
other  practitioners?  Thus,  the  idea  of  the 
Alternative  Assessment  Toolkit  was 
bom. 

The  Toolkit,  in  its  final  form,  includes 
assessment  materials  from  nearly  one 
dozen  participating  programs.  The  tools, 
along  with  instructions  on  their  use,  are 
divided  into  the  three  basic  categories  of 
assessment  Initial,  Ongoing,  and  Look- 
ing Back.  Within  these  sections  the  tools 
are  separated  by  program  to  stress  that 
all  of  these  tools  have  evolved  in  a 
particular  context 
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We  preferred 
using  "partner" 
rather  than 
"mentor",  because 
we  see  ourselves 
as  always  in 
development  with 
our  practice  and 
we  knew  we  were 
likely  to  learn  as 
much  through  the 
project  as  the  new 
participants. 


In  the  tall  of  1993,  as  the  Toolkit  and 
Component  Three  neared  completion, 
we  discussed  ways  to  continue  our 
work.  Certainly  the  Toolkit  would  be  of 
value  for  the  field,  but  we  still  had 
several  questions:  Could  the  value  that 
was  based  on  collaboration  be  main- 
tained and  broadened  to  more  pro- 
grams? Were  there  other  ways  our  work 
could  support  and  strengthen  the  way  of 
alternative  assessment  in  the  field  of 
adult  education?  In  what  ways  was  our 
work  replicable  in  other  programs? 

Gradually  the  idea  of  a mentoring 
project  evolved.  Such  a project  would 
allow  us  to  continue  our  own  develop- 
ment from  a new  perspective.  It  would 
broaden  the  network  of  adult  educators 
using  alternative  assessment.  Most 
importantly,  it  would  allow  us  to  ob- 
serve and  evaluate  firsthand  how  our 
materials  would  be  adapted  by  other 
programs. 

The  idea  excited  us,  and  it  interested 
the  DOE.  Throughout  Fall,  1993,  a core 
group  of  the  original  Component  Three 
now  facilitated  by  teacher /administrator 
Caroline  Gear  of  the  International 
Language  Institute  of  MA.,  Inc,  contin- 
ued to  meet  and  work  out  the  details. 

The  Partnership  Project,  as  we  named 
it,  would  pair  together  a member  of 
Component  Three  with  a practitioner 
who  wanted  to  use  and  develop  alterna- 
tive assessment  in  their  Hass  and/or 
program.  We  preferred  rising  "partner" 
rather  than  "mentor",  because  we  see 
ourselves  as  always  in  development 
with  our  practice  and  we  knew  we  were 
likely  to  learn  as  much  through  the 
project  as  the  new  participants.  Partner- 
ships would  be  designed  around  similar 
contexts  - beginning  ESL,  for  example 


and  similar  interests  - goal  setting,  self- 
evaluation,  portfolio,  etc.  Our  specific 
objectives  as  stated  in  our  formal  invita- 
tion to  participants  were  to  "disseminate 
the  Toolkit  to  a broader  audience,  to 
document  the  process  of  development 
and  adaptation  at  the  participating  sites, 
and  to  create  a broader  network  for 
alternative  assessment  within  the  state". 

From  the  very  beginning,  the  Partner- 
ship Project  was  designed  to  correspond 
with  the  demanding  schedule  of  adult 
education  practitioners.  The  length  of 
the  project  would  be  brief,  from  January 
to  June  of  1994.  The  technical  assistance 
time  of  the  partners  was  kept  to  a mini- 
mum (20  hours)  and  the  form  of  assis- 
tance - telephone  calls,  class  observation, 
etc  - was  left  to  the  discretion  of  the 
partners.  Emphasis  was  placed  on  the 
geographical  proximity  of  the  partners 
so  that  distance  of  travel  would  not  be 
an  obstacle.  Central  meetings  were 
necessary,  but  would  only  be  held  at 
the  beginning,  middle,  and  end  of  the 
process  - a reflection  of  the  Initial, 
Ongoing,  and  Looking  Back  steps  of 
the  alternative  assessment  process  (see 
Overview).  Caroline  Gear  and  I volun- 
teered to  coordinate  the  project. 

We  publicized  the  project  during  the 
fall  of  1993  - first  at  Network  93  (a 
statewide  Adult  Education  conference 
sponsored  by  the  Massachusetts  Coali- 
tion for  Adult  Education),  and  then  at  an 
alternative  assessment  class  provided  by 
SABES'  Western  Region  office.  Though 
initially  we  received  a lot  of  positive 
attention,  these  efforts  yielded  only  a 
few  formal  responses.  In  the  end,  word- 
of-mouth  publicity  was  our  best  re- 
source, and  yielded  a surprisingly 
diverse  membership. 
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New  participant  Pauline  O'Leary,  a 
teacher  at  ABCD's  Even  Start  Program  in 
Dorchester,  had  partnered  with  Barbara 
Krol-Sindair  from  Chelsea's  Intergenera- 
tional  Reading  Project. 

ESL  teacher  and  administrator  Lesly 
Desiree,  of  Dorchester's  Log  School, 
would  work  with  counselor  Henry 
Joseph  of  the  Haitian  Multi-Service 
Center,  also  of  Dorchester.  Dan  Wilson 
of  Boston's  North  End  Union  would 
focus  on  Beginning  ESL  with  Rudee 
Atlas  of  the  East  Boston  Harborside 
Community  Center.  Estelle  Williams, 

align  nf  fhn  NJnrfh  FnH  Union,  WOUld 

meet  with  Loretta  Pardi  of  Harborside 
regarding  her  mixed  level  GED  students. 
Widi  Sumary ano,  an  ESL  teacher  at 
Lutheran  Services  in  Springfield,  would 
be  partnered  with  Caroline  Gear  of  the 
International  Language  Institute  in 
Northampton.  Deirdre  McLaughlin  of 
Barnstable  County  House  of  Correction 
(ABE/ESL)  program  paired  with  Marty 
Tass-Richardson  of  Haverhill's  Commu- 
nity Action  Inc. 

Though  all  of  these  agencies  had  been 
involved  in  the  Component  Three 
GOALS  Project,  many  of  their  partici- 
pants were  new  to  our  collaborative. 
Henry  Joseph  of  HMSC,  Loretta  Pardi  at 
the  Harborside,  and  Marty  Tassi- 
Richardson  of  Community  Action  had 
developed  and/or  used  the  materials  at 
the  agencies,  but  none  had  participated 
directly  in  the  meeting  of  Component 
Three.  This  was  because  the  original  lead 
people  at  their  agencies,  after  a year  and 
a half  of  direct  involvement  in  the 
GOALS  Project,  were  only  to  pass  the 
baton  to  the  others  at  their  programs. 
Also  noteworthy,  Deirdre  and  Lesly  had 
participated  in  Component  Three,  but 


were  both  new  to  alternative  assessment 
and  did  not  feel  prepared  to  contribute 
materials  to  the  Toolkit.  They  wanted  to 
participate  as  mentees.  Of  the  partners, 
five  had  never  formally  used  alternative 
assessment  in  their  classrooms. 

Our  Initial  Meeting  was  held  at  the 
end  of  January  at  the  DOE's  new  space 
in  Malden.  (Though  our  scope  was 
statewide,  all  but  two  of  our  partners 
were  from  Eastern  Mass.)  Prior  to  this 
meeting,  participants  were  already 
assigned  partners  and  had  been  given 
the  task  of  touching  base  at  least  once,  to 
Hjcrncc  their  interests  2r.d  snsurs 
the  partnership  was  not  wrong  from  the 
start.  The  purpose  of  the  first  central 
meeting  was  for  the  different  pairs  to 
meet  each  other,  share  their  expecta- 
tions, and  to  discuss  together  the  goals 
and  expectations  of  the  project. 

January's  rough  weather,  however, 
prevented  almost  half  of  the  participants 
from  attending  - some  of  these  that  did 
attend  didn't  arrive  until  late  in  the 
meeting.  We  used  the  time  as  best  we 
could.  After  our  introductions  and 
preliminary  discussion,  we  were  joined 
by  Bob  Bickerton,  Director  of  the  newly 
reorganized  DOE  Bureau  of  Adult  and 
Community  Learning  Services  (formally 
the  Bureau  of  Adult  Education).  He 
spoke  of  the  DOE's  continued  interest  in 
investigating  alternative  assessment's 
role  in  a statewide  plan.  Because  of  this, 
he  said  the  DOE  was  interested  in 
funding  our  project  Though  this  was 
good  news,  die  partners  stressed  that 
they  were  in  favor  of  the  straightforward 
objectives  and  minimized  time  commit- 
ment of  the  Partnership  Project,  and  so 
were  in  favor  of  DOE  support  as  long  as 
it  didn't  increase  the  work  load.  By  the 


The  purpose  of 
the  first  central 
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the  different  pairs 
to  meet  each  other, 
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expectations  of 
the  project. 
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time  of  our  second  meeting,  DOE  had 
agreed  to  fund  the  project  by  supporting 
the  coordination  and  providing  stipends 
for  all  the  partners. 

Our  second  meeting  at  the  beginning 
of  April  was  a thrill.  Almost  everyone 
was  there.  In  three  tight  hours  we 
packed  individual  sharing,  updates, 
payment  info,  and  goals  setting.  This  last 
activity  was  vital,  since  we  had  only  two 
months  before  our  scheduled  final 
meeting  and  the  end  of  the  project. 

Deirdre  and  Marty  had  only  met  once 
briefly  before  the  second  meeting  (they 
had  only  been  paired  three  weeks  prior, 
after  Marty's  original  partner  had 
dropped  out).  Though  geographically 
far  apart  (Marty  was  on  the  North  Shore 
and  Deirdre  was  on  Cape  Cod)  and 
working  in  different  contexts  (a  commu- 
nity based  agency  and  a county  house  of 
correction),  they  had  already  identified  a 
common  area  of  interest:  the  ongoing 
assessment  of  their  students. 

Estelle  and  Loretta  had  met  several 
times  and  had  discussed  at  length  the 
motivational  difficulties  which  they  face. 
They  had  agreed  to  focus  on  ongoing 
tools  that  would  encourage  learners  to 
take  responsibility  for  their  learning  and 
increase  their  self  esteem. 

Lesly  and  Henry  had  been  pleasantly 
surprised  to  learn  that  their  programs 
were  only  a few  blocks  from  each  other 
and  that  they  serve  similar  populations. 
Lesly  had  already  been  using  the 
HMSC's  initial  and  ongoing  assessment 
tools  and  he  and  Henry  would  choose 
one  tool  or  area  as  a focus.  Widi  and 
Caroline  had  visited  each  other's  pro- 
grams and  observed  each  other's  classes. 
They  had  already  been  focusing  on  oral 
feedback  and  the  use  and  adaptation  of 


Ill's  Learner  Log. 

Dan  and  Rudee  had  met  and  dis- 
cussed their  needs  for  assessment. 
Rudee's  work  emphasized  student  self- 
assessment  and  Dan  had  chosen  this  as 
his  focus. 

Pauline  and  Barbara  had  their  plan 
from  the  very  beginning.  They  wanted  to 
create  a new  tool  that  would  focus  on 
parent-child  interaction.  They  wanted  to 
measure  the  academic  growth  of  the 
parents  through  structured  parent-child 
learning  activities. 

It  was  already  clear  that  with  eight 
weeks  left  in  the  project,  the  most  we 
could  expect  would  be  the  development 
and  limited  implementation  of  tools  and 
activities  in  the  partners'  chosen  areas. 

Participants  were  pumped  up  by  the 
end  of  the  meeting:  "a  good  use  of  time", 
said  one;  "great  to  hear  from  one  an- 
other", said  another.  "The  best  part  was 
the  sharing",  said  a third.  Practitioners 
were  invigorated  by  the  opportunity  to 
discuss  their  goals  and  processes  with 
others  working  on  some  of  the  same 
questions. 

Our  third  meeting  at  the  beginning  of 
June  came  all  too  soon.  All  of  our  pro- 
grams were  in  their  final  weeks.  All  of  us 
seemed  equally  frazzled  by  the  normally 
full-to-bursting  spring  schedule  of  our 
programs.  For  some,  the  burden  was 
greater  than  usual:  Caroline  had  to  teach 
extra  classes  because  of  funding  cuts; 
Lesly  was  in  the  same  boat.  Marty  had 
discovered  that  funding  had  been  cut  for 
her  program  and  it  was  closing  by  the 
end  of  the  month.  The  stories  were  all 
too  familiar,  and  as  adult  educators,  we 
had  heard  them  too  many  times. 

We  sat  down  with  our  sandwiches 
and  sodas  to  share  our  work,  look  at  our 
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goals,  and  see  how  we'd  done.  The  video 
camera  in  the  comer  of  the  room  was  a 
little  distracting  at  first,  but  we  soon 
were  caught  up  in  the  flow  of  the  meet- 
ing. Dan  passed  around  his  student  self- 
evaluation  forms.  We  all  enjoyed  the 
little  people  figures  used  for  fire  scale. 
Deirdre  and  Marty  passed  out  their  new 
intake  package  which  they  had  faxed  to 
one  another  for  final  revisions,  and  each 
program  had  adapted  theirs  slightly 
differently.  Pauline  discussed  the  effec- 
tiveness of  her  new  parent  evaluation 
tool  and  responded  to  comments  and 
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in  reviewing  and  considering  the  tools 
already  in  use  at  the  Log  School  and  in 
the  Toolkit.  Loretta  had  developed  a 
Record  of  Participation  that  would 
enable  both  learners  and  teachers  to 
evaluate  progress.  Widi  had  many 
examples  of  student  work  and  insights 
on  the  effectiveness  of  his  tools  and 
methods. 

After  our  sharing,  Bob  Bickerton 
visited  us  briefly  and  congratulated  us 
on  our  work.  He  also  asked  us  to  con- 
sider the  next  steps:  What  role,  if  any, 
might  alternative  assessment  play  in  a 
state  assessment  system?  What  roles,  if 
any,  did  we  intend  to  play?  We  consid- 
ered these  questions  as  we  considered 
our  own  next  steps:  participating  in 
Network  94,  meeting  after  publishing 
our  articles  in  the  fall,  organizing  a 
broader  workshop  for  the  fall,  perhaps 
continuing  our  partnerships  informally 
in  the  future. 

At  the  end  of  the  day,  the  feedback  (as 
in  a participatory  classroom,  we  hoped 
to  elicit  partner  feedback  throughout  the 
process)  was  mostly  positive:  coming 
together  is  wonderful,  the  variety  of  our 


work  is  impressive,  we  go  away  with  so 
many  ideas.  The  recurrent  lament  was 
lack  of  time,  but  we  all  knew  that,  with 
our  schedules  so  busy,  even  this  three 
hour  meeting  was  luxurious. 

How  had  we  done  on  our  objectives? 
We  had  broadened  the  network  of  adult 
educators  using  alternative  means  of 
assessment,  and  six  more  practitioners  at 
five  separate  agencies  in  Massachusetts 
had  developed  materials  and  practices. 
The  collaboration  that  was  started  in 
Component  Three  had  been  expanded  to 
include  a county  house  of  corrections 
and  an  Even  Start  Familv  Literacy 
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Program.  New  programs  joined  from 
Springfield  and  Barnstable,  as  well  as 
the  Boston  area. 

How  had  our  material  been  adapted? 
Lesly  borrowed  directly  from  the  Toolkit 
and,  without  much  adaptation,  was  able 
to  use  the  tools  with  his  students. 
Estelle's  GED  class  borrowed  and 
adapted  materials  from  Loretta.  Dan's 
ESL  class  had  taken  guidance  from 
Rudee's  work  to  develop  something 
new.  Widi  worked  closely  with  Caroline 
borrowing  and  developing  to  suit  the 
needs  of  his  students  and  also  got 
feedback  on  how  he  was  practicing 
assessment.  Pauline  and  Barbara  drew 
on  their  common  experience  to  create 
something  new  and  necessary  for 
Pauline's  class. 

What  was  apparent  for  everybody 
was  that  a lot  of  talking  and  thinking 
and  observing  had  taken  place.  This 
meeting  was  an  important  piece  of  it,  a 
chance  to  articulate  the  process  and 
reflect.  What  also  seemed  true  was  that 
we  had  just  gotten  started.  This  was  true 
in  that,  on  one  hand,  most  of  the  part- 
ners had  just  developed  their  tools,  or 
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had  had  the  opportunity  to  try  it  out 
only  a few  times.  Six  months  to  start  a 
partnership  and  begin  a whole  process 
of  assessment  was  only  just  enough  time 
to  get  going.  Clearly  for  partners  who 
had  worked  together  before,  such  as 
Pauline  and  Barbara,  or  for  those  who 
had  previous  experience  in  assessment, 
such  as  Widi  and  Deirdre,  the  project 
was  more  manageable.  Those  who  were 


freshest  had  more  ground  to  cover  in 
maybe  too  short  of  a time.  On  the  other 
hand,  as  one  participant  said,  "This  is 
only  the  beginning."  A six-month  project 
was  enough  to  draw  in  the  partners  to  a 
place  of  greater  knowledge  and  commit- 
ment to  alternative  assessment.  Many  of 
them  will,  I hope,  continue  their  investi- 
gation of  alternative  assessment  and 
how  it  works  in  their  classroom. 
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ith  the  GOALS  Project  wind- 
ing down  and  the  Toolkit 
nearing  completion,  many  of 
the  participants  in  the  Authentic  Assess- 
ment Component  felt  that  their  work 
was  not  finished.  We  all  felt  strongly  that 
we  didn't  want  people  to  just  take  the 
Toolkit  and  photocopy  the  tools  without 

trulv  understanding  the  what.  whv.  and 
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for  whom  the  tool  was  designed.  What 
better  way  of  disseminating  our  tools 
than  by  setting  up  a mentoring  project 
that  would  pair  members  of  the  Authen- 
tic Assessment  group  with  partners  that 
were  interested  in  adapting  tools  from 
the  Toolkit  to  their  programs? 

I felt  fortunate  to  be  both  a coordina- 
tor and  a mentor  of  the  6-month  Partner- 
ship Project  that  ran  from  January  to 
June  of  1994.  It  was  a luxury  to  be  com- 
pletely focused  on  one  aspect  of  adult 
education:  Authentic  assessment  in  the 
ESL/ABE  classroom.  I was  partnered 
with  Widi  Sumaryano  who  teaches  ESL 
at  Lutheran  Services  in  West  Springfield. 
Very  early  in  our  partnership,  we  devel- 
oped a sense  of  trust  between  us,  and  we 
created  a truly  wonderful  working 
relationship.  This  article  will  share  our 
adventures  and  the  unanticipated  out- 
comes of  the  project. 

Because  of  an  ice  storm,  Widi  and  I 
were  unable  to  attend  the  initial  meeting 
of  all  the  participants  at  the  DOE  in 
Malden.  We  arranged  to  have  our  own 
initial  meeting  at  the  International 
Language  Institute  of  MA.,  Inc  (ILI)  to 


discuss  the  logistics  of  the  project.  We 
had  already  spoken  on  a number  of 
occasions  and  knew  that  we  would  be 
able  to  work  together.  At  our  initial 
meeting,  we  also  talked  about  how  we 
viewed  assessment,  what  types  of 
assessment  we  had  previously  done, 
and  what  types  of  assessment  we  were 
i icing  in  our  prograins.  Ws  discuss sd  the 
Learner  Log  (see  AiA,  Fall,  1993)  that  ILI 
uses  and  decided  to  limit  our  scope  of 
work  and  concentrate  on  two  areas: 
weekly  written  evaluation  forms  and 
oral  feedback. 

We  talked  about  how  ILI  uses  oral 
feedback,  and  I asked  Widi  to  try  the 
following  steps  in  his  class: 

• Elicit  from  students  the  activities  that 
they  did  in  the  class  and  write  what 
they  say  on  the  board 

• Ask  which  activities  were  the  most 
helpful  in  learning  English 

• Ask  which  activities  were  the  least 
helpful  in  learning  English 

• Ask  which  activities  they  would  like 
to  see  more  of 

• Ask  which  activities  they  would  like 
never  to  see  again 

Oral  feedback  needs  to  be  treated  as  a 
process  that  the  teacher  does  consis- 
tently at  the  end  of  class,  or  after  a 
certain  activity  that  is  new  to  the  stu- 
dents and  an  activity  on  which  the 
teacher  wants  immediate  feedback. 

From  the  start,  teachers  need  to 


The 

Partnership  | { 
Prefect 


by 

Caroline  Gear 


International  Language 
Institute  of  MA,  Inc 
Northampton,  MA 


Vohnie  7:  Fall  1994 


ADVENTURES  IN  ASSESSMENT 


Oral  feedback 
is  an  ongoing 
process.  As  both 
teachers  and 
students  become 
more  comfortable 
with  oral  feedback, 
teachers  get  better 
at  facilitating  it, 
and  students 
get  better  at 
responding  to  it. 


depersonalize  the  feedback  so  that 
students  understand  that  they  are 
commenting  on  the  activity  and  not  the 
teacher.  Teachers  need  to  be  clear  as  to 
why  they  are  doing  oral  feedback  in  the 
class,  and  students  need  to  understand 
why  the  teacher  is  taking  class  time  to 
ask  them  questions  about  the  class.  By 
doing  oral  feedback,  students  are  more 
prepared  for  weekly  written  evaluations 
that  ask  for  feedback  about  the  class,  and 
if  the  students  are  satisfied  with  their 
progress  in  the  class.  Oral  feedback  is  an 
ongoing  process.  As  both  teachers  and 
students  become  more  comfortable  with 
oral  feedback,  teachers  get  better  at 
facilitating  it,  and  students  get  better  at 
responding  to  it. 

We  set  up  a time  for  Widi  to  observe 
one  of  our  classes  at  ILL  The  timing  was 
perfect  as  it  was  the  start  of  the  program, 
and  he  would  be  able  to  see  how  stu- 
dents were  first  exposed  to  the  idea  of 
feedback  and  the  Learner  Log.  Widi 
suggested  that  he  observe  the  class  at  the 
mid-term  and  at  the  end  of  the  program 
so  that  he  could  fully  understand  the 
Intake,  Ongoing,  and  Looking  Back 
assessment  at  ILL  Widi  chose  two 
samples  of  weekly  evaluations  from  ILI 
to  review  and  try  in  his  classes. 

After  Widi  had  observed  a class  at  ILL 
I asked  him  if  he  had  learned  anything 
from  observing  the  instructor  doing  oral 
feedback.  "Yes.  I was  able  to  observe 
what  you  had  coached  me  to  do  and 
what  I have  tried  several  times.  I think 
the  idea  of  having  oral  feedback  as  a 
ritual  thing  is  very  good  and  I'll  continue 
to  do  that  because  it  has  a sense  of 
giving  a nice  closure  to  the  class.  The 
day  is  dosed  with  this  relaxing  and 
sharing.  There  is  an  opportunity  for 


students  to  say  what  they  like  and  by 
doing  that  I feel  more  certain  of  what  I 
will  do  tomorrow.  I can  be  more  pre- 
pared when  doing  my  course  planning 
knowing  which  activities  students  like  or 
don't  like." 

Widi  was  very  enthusiastic  about  the 
dass  and  we  began  talking  about  other 
aspects  of  the  dass.  He  mentioned  that 
he  had  seen  familiar  activities  that  he 
hadn't  been  using,  and  that  he  was 
looking  forward  to  trying  these  activi- 
ties. He  specifically  commented  on  how 
the  teacher  was  using  a Total  Physical 
Response  (TPR)  activity  in  the  class.  He 
used  TPR  in  his  class,  but  he  noted  that 
she  had  incorporated  the  activity  to 
indude  the  other  skill  areas  and  that  he 
felt  that  he  could  and  should  be  doing 
this  inhisclass. 

During  the  following  months  of  the 
project,  I saw  a change  in  Widi  Through 
our  telephone  conversations  I could  hear 
him  truly  begin  to  understand  the 
benefit  of  doing  oral  feedback  with  his 
students:  'Tm  getting  a lot  more  feed- 
back from  my  students  when  I ask  them 
about  the  class.  It's  a completely  differ- 
ent style  of  teaching  than  what  I have 
been  doing  in  my  class.  I never  had  a 
chance  to  do  this  before.  Before  I got 
involved  with  this  project,  I decided 
what  to  cover  in  the  class.  Now  I feel 
that  the  students  are  choosing  what  they 
would  like  to  do." 

"I  realize  that  doing  oral  feedback  is  a 
process  where  the  teachers  and  students 
become  more  effective  with  feedback  as 
time  goes  by.  I feel  that  this  is  a very 
good  notion  that  feedback  is  about  the 
activity — about  what  we  are  doing  in 
the  class  and  not  who  we  (the  teachers) 
are.  I'm  interested  in  how  I can  facilitate 
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feedback  and  how  I will  get  the  students 
to  think  about  the  activity  and  the  value 
of  the  activity." 

One  of  the  major  factors  of  our  success 
as  a mentoring  team  was  due  to  the  fact 
that  we  were  the  only  participants  from 
western  Massachusetts.  All  the  whole 
group  meetings  had  been  scheduled  in 
Boston  to  accommodate  the  participants 
from  the  Eastern  part  of  the  state.  Widi 
and  I had  more  of  an  opportunity  to  get 
to  know  each  other  during  the  project  as 
our  travel  time  to  and  from  our  meetings 
was  the  forum  to  talk  about  assessment, 
and  also  to  delve  into  other  areas  of  our 
programs.  Our  "car  talk"  ranged  from 
E5L  techniques  in  the  classroom  to 
teacher  training;  from  staff  development 
to  how  to  deal  with  a difficult  program 
manager. 

As  the  project  went  on,  Widi  became 
more  confident  with  feedback  and 
evaluation  in  his  classes.  As  he  saw  the 
benefits  from  doing  feedback,  he  had 
also  been  sharing  this  process  with 
another  ESL  teacher  in  the  program. 

Widi  had  asked  the  teacher  to  do  oral 
feedback  with  his  class,  but  what  ensued 
caused  him  to  stop  dead  in  his  tracks. 

The  teacher  had  tried  oral  feedback 
with  his  class  and  the  feedback  session 
became  a negative  experience  for  him;  a 
student  angrily  accused  the  teacher  that 
her  needs  were  not  being  met  by  the 
way  the  class  was  taught.  The  teacher 
was  angry  with  the  feedback  and  an- 
noyed at  Widi  for  getting  him  to  do 
feedback  in  his  class. 

Widi  recounted  to  me  that  he  had  met 
with  the  teacher  after  the  class  and  told 
him  that  most  likely  the  student  had  had 
these  feelings  pent  up  and  finally  had 
the  opportunity  to  vent.  Widi  suggested 


that  the  instructor  do  more  oral  feedback 
so  that  students  wouldn't  bottle  things 
up.  As  Widi  related  this  incident  to  me, 

I found  this  situation  very  interesting; 
while  I was  coaching  Widi,  he  was 
coaching  his  co-teacher.  The  episode 
with  Widi  and  his  co-teacher  demon- 
strates the  importance  of  establishing 
ground  rules,  a trusting  environment, 
and  a belief  that  the  feedback  is  to 
benefit  all  parties  involved  when  enter- 
ing the  realm  of  peer  coaching. 

Teachers  need  to  understand  not  only 
why  they  are  doing  feedback,  but  they 
also  need  to  be  coached  on  kczv  to  facili- 
tate the  discussion.  It's  always  difficult 
for  the  ego  when  someone  has  a negative 
comment  about  the  class.  It  is  hard  not  to 
think  that  the  comment  is  directed  at 
you,  the  teacher.  Teachers  need  to  be 
coached  in  facilitating  feedback,  and 
teachers  and  students  alike  need  to 
understand  that  when  they  talk  about 
the  class  they  are  talking  about  the 
activities,  and  not  the  personality  of  the 
teacher.  We  need  to  depersonalize  the 
feedback  by  focusing  on  the  activities 
rather  than  the  teacher. 

After  the  episode,  Widi  discussed 
why  he  had  felt  that  his  session  with 
the  teacher  did  not  go  well;  that  it  was 
almost  an  authority-subordinate  conflict 
He  said  that  he  wished  that  he  had 
posed  more  questions  to  the  teacher 
about  the  class  rather  than  talking  so 
much  about  the  class  and  what  the 
teacher  could  have  done  in  the  class. 

He  also  mentioned  that  if  he  had  done 
more  peer  evaluation  with  the  instructor, 
maybe  the  instructor  would  have  felt 
better  about  Widi's  suggestions. 

At  the  beginning  of  the  Partnership 
Project,  Widi  had  mentioned  that  he 
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hadn't  felt  comfortable  giving  feedback 
to  co-teachers.  He  felt  that  they  also  had 
experience,  and  he  felt  insecure  in  giving 
them  feedback.  Widi  was  carrying  over 
his  ideas  of  evaluation  to  other  areas.  He 
was  not  only  doing  evaluation  with  his 
class,  but  he  was  evaluating  himself.  He 
wasn't  feeling  insecure  about  doing 
feedback  with  the  instructor,  but  had 
moved  on  to  focusing  on  how  he  could 
improve  his  ability  to  give  feedback  and 
evaluation  to  his  teachers. 

Widi  and  I talked  about  working  with 
other  teachers  and  we  came  up  with 
some  strategies: 

• Do  more  peer  evaluations  with  the 
instructor 

• Make  sure  the  instructor  observes 
Widi  doing  feedback  and  discuss 
what  the  instructor  saw 

• Pose  questions  to  the  instructor  and 
try  to  get  the  instructor  to  reflect/ 
comment  on  the  class  before  Widi 
comments  on  it 

Widi  and  I both  felt  that  we  had  been 
spending  a lot  of  time  on  teacher  train- 
ing and  that  maybe  we  were  getting 
side-tracked.  We  decided  to  focus  more 
on  the  weekly  written  evaluations  in 
subsequent  meetings,  but  still  touch  base 
with  what  was  going  on  with  Widi  and 
his  co-teacher. 

One  of  the  important  steps  in  estab- 
lishing a good  mentoring  relationship  is 
for  the  mentors  to  visit  each  other's 
program  as  many  times  as  possible.  It 
was  extremely  beneficial  to  our  partner- 
ship to  see  the  physical  environment, 
meet  the  people  and  students,  and 
discuss  with  staff  and  students  what  the 
Partnership  Project  was  about.  Having 
staff  and  students  of  the  programs 


understand  our  project  validated  our 
belief  that  this  was  an  important  project. 

Along  with  oral  feedback,  Widi  had 
adapted  our  written  weekly  evaluation 
forms  (Learner  Logs)  to  his  program,  and 
found  that  as  the  students  got  vised  to 
the  forms,  they  began  to  write  more 
about  the  class  and  what  they  wanted  to 
see  more  of  in  the  class. 

In  May  I observed  Widi  with  the 
intention  of  giving  him  feedback  about 
how  he  was  doing  oral  feedback.  I 
observed  the  whole  class,  and  began 
taking  notes  on  what  he  did  in  his  class. 
Over  coffee  we  talked  about  the  class.  I 
asked  him  if  he  wanted  some  feedback 
on  the  whole  class,  rather  than  just  m 
how  he  was  doing  oral  feedback.  We 
reviewed  the  specific  activities  of  the 
class,  and  I asked  Widi  if  there  was 
anything  in  the  class  that  he  didn't  feel 
comfortable  with.  We  also  talked  about 
why  he  had  chosen  an  activity,  and  then 
ended  our  discussion  with  suggestions 
for  future  classes. 

There  is  not  a lot  of  staff  development 
or  support  for  ESL  instructors  in  his 
program,  and  he  was  eager  to  hear  what 
another  ESL  instructor  had  to  say  about 
his  class.  When  he  had  been  observed 
before  in  his  program,  it  was  by  an 
administrator  with  no  experience  in 
teaching  ESL. 

This  was  the  first  time  that  I had  given 
feedback  to  a teacher  that  was  not  in  our 
school.  It  seemed  easier  as  we  had  been 
working  closely  over  a period  of  time 
and  had  developed  a feeling  of  trust.  I 
felt  that  it  was  important  to  our  relation- 
ship for  Widi  to  observe  me  in  the  class 
and  give  me  feedback  about  the  activi- 
ties he  had  observed.  This  kept  our 
relationship  on  an  even  keel  and  demon- 
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strated  how  we  were  learning  from  each 
other. 

The  partnership  was  successful 
because  everything  we  focused  on  was 
modeled  for  Widi.  Oral  feedback  with 
students,  oral  feedback  with  peers,  and 
classroom  activities  all  were  set  up  for 
Widi  to  observe  and  comment  on.  Widi 
understood  that  what  we  were  focusing 
on  in  this  partnership  wasn't  "the  only 
way"  of  giving  feedback  to  his  students 
and  to  his  teachers,  but  he  saw  what  we 
were  doing  as  something  useful  and 
made  the  choice  to  extend  what  he  was 
learning  about  assessment  into  other 
areas.  This  enabled  Widi  to  grow  profes- 


sionally both  inside  and  outside  of  the 
classroom. 

The  Partnership  Project  ended  in  June, 
but  Widi  and  I continued  to  meet  with 
each  other.  Not  only  did  we  continue 
our  discussions  about  feedback  and 
evaluation,  but  we  also  met  to  discuss 
our  reflective  pieces  for  Adventures  In 
Assessment.  This  added  piece  made  us 
both  realize  that  authentic  assessment  in 
the  classroom  and  writing  for  the  journal 
are  on  parallel:  They  both  are  a process, 
require  constant  revision,  and  share  a 
feeling  of  never  being  the  perfect  piece 
or  the  perfect  tool  to  measure  progress. 
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Authentic  Assessment 
from  Another  Angle 


by 

Widi  Sumaryano 


Lutheran  Social 
Services, 

West  Springfield,  MA 


After  graduating  from  Sanata 
Dharma  Teacher  Training 
^ College,  I taught  ESL  in 
Indochinese  refugee  camps  in  Indonesia 
and  Thailand.  In  the  Indochinese  refugee 
training  programs,  teachers  from  many 
cultural  backgrounds  teach  classes  of 
refugee  students  to  promote  the 
refugee's  initial  resettlement  and  self- 
sufficiency  in  the  United  States.  Teachers 
work  with  refugee  students  in  the  areas 
of  English  language,  cultural  orientation, 
work  orientation  and  preparation  for 
American  secondary  schools.  There  are 
intensive  pre-service  and  on-going  in- 
service  teacher  trainings  to  promote 
teacher  independence,  developing  a 
teacher's  ability  to  make  informed 
choices  about  teaching. 

Six  years  later,  upon  coining  to  the 
US.,  I got  a part-time  teaching  job  with 
Lutheran  Social  Services.  I was  the  only 
teacher  and  was  assigned  to  revive  the 
dormant  ESL  program  for  refugees  from 
the  former  U.SS.R  The  director  told  me, 
"This  is  your  class  list  and  the  address  of 
the  church  where  the  classroom  is."  I 
called  my  Mends  to  ask  if  they  knew 
anything  about  Russian  culture.  With  a 
few  hints  from  them  I met  my  students 
the  next  day.  It  was  quite  challenging  to 
have  to  work  alone  with  minimal  guide- 
lines, no  available  pre-service  nor  in- 
service  trainings  at  all. 

I was  able  to  manage  the  program, 
however,  then  we  got  a grant  that  incor- 
porated employment  services  and  ESL. 


The  funder  mandated  that  we  put  more 
emphasis  on  employment-oriented  ESL. 
When  we  hired  more  teachers,  I was 
assigned  to  be  the  ESL  coordinator.  At 
first,  I did  not  feel  comfortable  doing  it 
because  I thought  that  every  teacher 
must  already  "know"  how  to  teach.  T did 
not  want  to  tell  other  teachers  what  to 
do.  I did  not  even  want  to  check  what 
they  were  doing.  I assumed  all  teachers 
could  function  without  supports  such  as 
orientation,  pre-service/in-service 
trainings  as  I had  done  in  the  U.S. 

However  there  was  a high  turnover  in 
my  program.  I felt  really  sorry  for  a few 
teachers  who  got  burned-out  and  left  my 
program.  I felt  terribly  guilty  for  not 
being  able  to  help  them  in  dealing  with 
their  challenges.  Since  then  I have  been 
encouraging  my  colleagues  to  attend 
conferences  and  workshops  through 
SABES,  MATSOL  and  other  professional 
affiliations.  After  attending  workshops 
and  conferences,  we  feel  energized  to 
apply  some  new  teaching  activities  that 
we  have  learned'.  I became  more  enthusi- 
astic and  a little  bit  more  creative  in  my 
planning  and  teaching,  but  it  did  not 
stay  long.  A few  months  later  things  got 
"mundane".  I became  the  "dead  wood" 
teacher,  the  one  that  always  did  things 
that  I was  familiar  with,  easy  prepara- 
tion or  no  preparation  at  all,  since  I 
already  had  the  materials  and  handouts 
I created  last  class.  So  I basically  did 
things  based  on  what  I had,  what  I liked, 
and  what  I knew  worked.  Everything 
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started  from  "I".  Nobody  really  super- 
vised me,  told  me  whether  I did  things 
effectively  or  not.  The  feedback  that  I got 
was  only  from  my  students.  They  did 
not  say  anything  directly,  but  I occasion- 
ally could  see  from  their  facial  expres- 
sions when  they  were  not  satisfied  with 
the  lesson  for  the  day. 

Unfortunately,  that  simple  feedback 
did  not  give  me  enough  clues  of  what 
they  wanted  or  what  to  do  next  Some 
students  channeled  their  feedback 
indirectly  through  other  staff  members. 
Unfortunately  often  times  other  staff 
members  used  the  feedback  as  an  instru- 
ment to  criticize  the  ESL  teachers. 

Then  I attended  a mini-course  on 
Authentic  Assessment  facilitated  by 
Lindy  Whiton  from  SABES.  This  course 
gave  me  a broader  view  about  evalua- 
tion, assessment,  feedback,  and  program 
development.  The  group  was  small  and 
participants  were  enthusiastic  enough  to 
share  what  they  were  doing  and  had 
developed.  Guest  speakers/facilitators 
came  from  various  programs.  Among 
the  workshops  I attended  was  one  about 
"Learner's  Logs,"  facilitated  by  Caroline 
Gear  from  International  Language 
Institute  of  Massachusetts,  Inc.  We 
explored  all  different  kinds  of  assess- 
ment tools  from  intake  and  on-going  to 
mid/post  assessment.  At  the  end  of  the 
mini-course,  I was  asked  if  I would  be 
interested  in  a mentoring  project  I was 
interested  even  though  I did  not  have  a 
clue  what  it  was  going  to  be. 

I got  involved  mainly  because  I 
realized  that  I needed  to  broaden  my 
understanding  of  assessment  to  develop 
my  program.  My  knowledge  and  experi- 
ence in  assessment  was  limited  to  stan- 
dardized assessment,  such  as  the  Basic 


English  Skill  Test  (BEST),  Michigan  Test, 
etc.  I liked  the  idea  of  using  Authentic 
Assessment  tools  and  I wanted  to  know 
more  about  them.  I decided  to  sign  up 
for  the  Authentic  Assessment  Partner- 
ship Project/ AAPP  and  chose  Caroline 
Gear  as  my  mentor. 

I think  die  expectation  was  that, 
through  this  project,  more  authentic 
assessment  tools  would  be  created/ 
refined  and  disseminated  to  enhance 
program  development,  as  an  alternative 
to  the  existing  commonly  used  standard- 
ized assessment.  The  goal  of  this  project 
was  building  professional  networks,  and 
researching  authentic  assessment  tools. 
My  goal  was  to  get  expert  coaching  in 
adopting/adapting  authentic  assessment 
tools  that  were  suitable  for  my  program. 
My  strong  focus  was  to  see  to  what 
extent,  and  in  what  ways,  students 
themselves  buy  into  the  assessment 
process  and  make  it  their  own. 

At  first  Caroline  and  I talked  about  the 
project  itself,  then  discussed  trying  the 
Learner  Log  in  my  class.  The  next  day,  I 
tried  an  assessment  tool  where  students 
made  a list  of  the  activities  that  we  had 
done  and  assigned  a value  to  them.  It 
was  difficult  for  my  students  to  name 
the  activities.  I shared  this  problem  with 
my  mentor,  and  she  encouraged  me  to 
try  oral  feedback  a few  minutes  before 
class  ended,  where  learners  are  invited 
to  voice  their  opinions  about  the  class. 

To  get  a better  understanding  of  how  to 
facilitate  this  assessment,  I observed 
classes  at  ILL  I invited  my  mentor  to 
visit  my  class  and  we  had  a follow-up 
discussion.  At  the  discussion,  we  real- 
ized that  there  was  a need  to  assist  the 
students  in  articulating  their  opinions  so 
they  would  not  just  try  to  please  the 
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teacher  by  saying  that  all  activities  were 
good  and  everything  was  fine.  My 
mentor  assured  me  that  by  doing  this 
assessment  consistently  it  would  eventu- 
ally help  the  students  to  be  more  clear.  I 
came  back  to  ILI  a month  later  to  visit 
the  class  I observed  before.  I wanted  to 
see  the  students'  progress  in  their  ability 
to  participate  in  oral  feedback  assess- 
ment After  the  observation,  my  mentor 
and  I discussed  the  class  I had  just 
observed,  and  she  gave  me  a nicely 
typed  feedback  as  a follow-up  from  the 
observation  she  did  the  week  before. 

Over  the  course  of  the  project  we  met 
with  other  Partnership  teams  several 
times,  updated  our  activities  and  dis- 
cussed the  next  steps.  In  a meeting  at  the 
DOE  office  in  Malden,  we  shared  the 
tools  that  were  adapted  or  newly  created 
by  participants.  In  one  of  these  meetings. 
Bob  Binkerton,  the  state  Director  of 
Adult  Basic  Education,  joined  us  to 
converse  about  authentic  assessment  in 
contrast  with  standardized  assessment 
in  regard  to  accountability.  Since  then  I 
have  been  thinking  about  it.  I have 
questioned  myself: 

• What  is  accountability? 

• Who  should  be  accountable? 

• To  whom  should  I be  accountable? 

• Why  should  I or  the  students  be 
accountable? 

• When  should  we  be  accountable? 

• How  do  I measure  how  well  students 
have  acquired  English  and  are  able  to 
function? 

I am  still  searching  for  the  answers, 
and  somehow  I feel  that  I will  find  them 
through  my  experimentation  with 
authentic  assessment 
I have  tried  to  facilitate  oral  feedback 
consistently  a few  minutes  before  class 


ends.  I have  tried  the  various  forms  of 
weekly  assessment  as  well  as  mid-cycle 
and  end-of-cycle  assessment.  I found  it 
interesting  that  some  students  highly 
valued  some  activities  that  1 thought 
nobody  would  like,  because  I did  not 
value  them  myself.  One  week  I ne- 
glected listening  activities  and  ultimately 
concluded  that  particular  skill  was  no 
longer  needed  since  I assumed  that  my 
students  had  always  understood  me 
with  ease.  I was  wrong,  as  some  stu- 
dents pointed  out  at  the  oral  feedback; 
they  needed  listening  practice. 

Oral  feedback  gives  a sense  of  closure 
to  the  session  or  day.  Before  I tried  oral 
feedback,  whenever  I finished  my  class,  I 
felt  like  there  was  a big  question  "What's 
next?"  I had  to  think  hard  for  the  an- 
swer. Now,  after  the  class  I feel  ener- 
gized. I know  what  my  students' needs 
are  for  the  next  day  and  can  plan  activi- 
ties that  help  them  learn.  Oral  feedback 
also  gives  the  students  the  chance  to  ask 
for  more  explanation  about  things  that 
were  not  quite  clear  to  them.  This  ritual 
of  doing  the  oral  feedback  improves  the 
ability  of  students  to  do  the  written 
weekly  learning  log. 

I was  surprised  to  see  how  much 
more  articulate  some  of  the  students 
became,  compared  with  the  previous 
month.  They  were  able  to  individually 
value  each  activity  according  to  their 
learning  styles,  and  to  mention  the 
things  that  they  would  like  to  learn  more 
about. 

From  this  Authentic  Assessment 
Partnership  Project,  I learned  a lot  not 
only  about  assessment  tools.  The  unex- 
pected outcome  in  the  process  of  trying 
these  new  assessment  tools  was  that  I 
learned  valuable  peer  coaching  training 
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and  giving  feedback;  how  to  observe, 
compliment  and  provide  suggestions. 
We  expanded  our  discussion  into  a 
larger  scope.  For  example  we  discussed 
Jazz  Chants  and  the  various  ways  to  use 
them  as  classroom  activities.  I learned 
about  empowering  students  by  encour- 
aging them  to  participate  more  actively 
in  oral  feedback.  From  observing  other 
classes,  I recalled  about  classroom 
management,  such  as  being  relaxed,  full 
of  laughter,  delegating  simple  tasks  to 
students,  cooperative  learning  and  also 
about  other  teaching  techniques,  differ- 
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index  cards,  books,  etc.  Most  of  the 


things  I saw,  I might  have  been  familiar 
with  before,  but  somehow,  the  fact  that  I 
saw  them  again  refreshed  me.  Even  the 
time  we  spent  in  the  car  on  the  round 
trip  to  Malden,  MA  provided  me  with 
lots  of  learning;  we  had  three  hours  to 
discuss  workplace  policy,  program 
development,  even  personal  matters. 

I felt  so  lucky  that  Caroline  went 
beyond  assessment  in  this  partnership. 
We  expanded  our  discussion  into  a 
larger  arena.  Looking  back  at  the  goals  I 
set  initially,  I think  I have  accomplished 
most  of  them.  Thanks  to  Caroline  Gear, 
Paiii  Tmrjiv!  Bob  BirJcsrton  for  2 job 
well  done. 
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Assessing  All  Things  that 
Make  A Student  Teachable 


by 

Loretta  Pardi 
Estelle  Williams 


Harborade  Community 
Center 

East  Boston,  MA 

& 

North  End  Union 


Boston,  MA 


During  the  Partnership  Project, 
Loretta  Pardi  of  Harborside 
Community  Center  and  Estelle 
Williams  of  the  North  End  Union 
worked  together  to  look  at  assessment 
that  went  beyond  the  content  areas  of 
the  GED  exam  and  look  at  how  assess- 
ment fit  into  motivating  students  to  take 
responsibility  for  their  own  learning. 

We  were  originally  interested  in  the 
Partnership  Project  because  we  expected 
it  would  help  us  as  teachers.  Teachers  do 
not  often  get  a chance  to  observe  or  work 
with  other  teachers,  so  we  saw  the 
Partnership  Project  as  a good  opportu- 
nity to  work,  talk  and  share  around  a 
common  concern,  in  this  case,  assess- 
ment Estelle  was  a first  year  GED 
teacher  and  Loretta  has  been  teaching 
GED  for  three  years  and  had  been 
involved  in  alternative  assessment  for 
the  same  period. 

During  this  project  we  hoped  to  share 
our  experiences  and  concerns  around 
teaching  and  assessment  in  a GED 
context.  We  also  hoped  we  would  have 
the  opportunity  to  try  new  techniques  or 
create  whatever  was  needed  to  assess 
our  students'  development  and  achieve- 
ment. As  a goal,  we  hoped  to  feel  more 
confident  handling  the  questions  of 
assessment  and  student  progress. 

At  our  first  meeting,  we  decided  to 
address  an  issue  that  seemed  most 
pressing  for  Estelle,  namely  finding  an 
easy  and  accurate  way  to  measure 
progress  in  math.  At  that  meeting 


Loretta  shared  the  assessment  tools  she 
had  been  using  to  assess  student 
progress  in  math  and  the  GED  content 
areas.  These  tools  were  adapted  from 
previous  Adventures  in  Assessment 
volumes,  as  well  as  teacher-made  tools 
from  her  program.  Loretta  and  Estelle 
looked  them  over  and  chose  one  to  try  to 
adapt.  During  the  discussion  that  day 
we  also  realized  that,  although  we  were 
from  different  sites  and  had  different 
populations,  we  had  many  of  the  same 
classroom  issues  and  experiences.  Our 
students  had  a lot  in  common.  The  ways 
our  students  survived  outside  the 
classroom  (e.g.  lying,  cheating,  drugs, 
etc.)  prevented  them  from  participating 
in  the  group  atmosphere  of  the  class- 
room. They  also  had  issues  relating  to 
peers  and/or  their  teachers.  Students 
were  constantly  challenging  us  as  teach- 
ers and  testing  our  ability  to  set  limits 
and  maintain  control  within  the  class- 
room. 

When  Spring  was  at  its  finest,  our 
attendance  was  at  its  lowest.  It  was 
obvious  through  our  conversations  that 
our  classroom  situations  were  not 
improving.  We  realized  the  students 
lacked  commitment  to  their  achieve 
ment,  class,  peers  and  teachers.  They 
also  lacked  belief  in  their  ability  to 
achieve  goals  and  did  not  feel  responsi- 
bility for  their  own  progress  or  learning. 

After  careful  thought  we  concluded 
that,  perhaps  we  were  concentrating  too 
much  of  our  effort  on  tracking  the 
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progress  of  students  academically  within 
the  content  areas  of  the  GED.  Students 
were  not  yet  committed  to  their  aca- 
demic success  and  did  not  view  social 
participation  in  the  group  setting  of  a 
class  to  be  a realistic  expectation.  We 
regrouped  and  committed  ourselves  to  a 
goal  of  assessing  all  things  that  make  a 
student  teachable. 

We  proceeded  by  asking  ourselves  the 
following  questions.  (We  did  not  expect 
to  answer  all  these  questions  at  once.) 

• How  do  we  assess  our  effectiveness  as 
teachers  of  GED  with  an  individual? 

• Do  we  need  to  assess  our  students' 
ability  to  learn  and  achieve  within 
normal  boundaries? 

• Should  our  students  be  held  respon- 
sible for  understanding  and  abiding 
by  the  same  code  of  ethics  as  other 
school  age  students,  e.g.  K-12? 

• Should  we  teach  these  (code  of  ethics) 
in  Adult  Education? 

• Does  the  class  schedule  meet  the 
needs  of  the  students? 

• Are  we  meeting  the  students' needs 
by  having  a learning  environment  that 
is  encouraging  supportive  and  fair  to 
all? 

We  wanted  to  develop  a system  that 
would  encourage  students'  commitment, 
beliefs  and  fairness  to  the  group,  as  well 
as  a tool  to  increase  their  self-esteem  and 
encourage  them  to  take  responsibility  for 
their  learning.  To  do  this  in  a timely 
fashion,  we  reviewed  all  the  assessment 
tools  we  had  from  our  own  collections 
and  Ae  'Toolkit"  revised  a few,  and  put 
them  together.  We  started  with  a daily 
journal  that  asked  students  to  respond  to 
several  questions  that  would  give  stu- 


dents the  opportunity  to  address  an 
experience  in  the  classroom. 


This  did  not  work.  Students  would 
talk  about  these  topics  in  classes  but 
were  not  willing  to  put  anything  in 
writing.  It  was  also  time  consuming. 

We  had  many  discussions  concerning 
how  else  we  could  encourage  students  to 
monitor  their  own  progress  in  abilities 
not  directly  related  to  the  content  areas 
of  the  GED  test  itself.  Many  issues  are 
vital  not  only  to  successful  GED  study 
but  also  to  other  areas  in  our  students' 
lives,  such  as  the  search  for  employment 
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students  through  conferences  and 
informally  during  class  time  were  time 
organization,  goals  setting,  and  sus- 
tained commitment  and  actions  towards 
short  and  long  term  goals. 

We  wanted  to  develop  a tool  to  enable 
students  to  chart  their  own  progress  in 
these  areas.  After  much  thought  we  then 
came  up  with  our  Record  of  Participa- 
tion (ROP)  (see  Figure  1).  It  would  help 
students  recognize  what  they  have 
accomplished  and  what  they  need  to  do 
in  order  to  attain  goals. 

Our  Record  of  Participation  asked  the 
instructor  to  evaluate  student  progress 
and  for  the  student  to  evaluate  his/her 
own  progress.  Because  the  record  is 
designed  to  be  completed  on  a regular 
basis,  the  student  will  reflect  on  his/her 
activities  in  a timely  fashion;  s/he  can 
recognize  the  quality  of  his/her  partici- 
pation for  the  past  week,  and  then 
hopefully  build  on  that  success  or  ad- 
dress specific  issues  in  the  coming 
weeks.  Plans  for  the  future  would  be 
documented  here  and  show  long  and 
short  term  goals. 

This  system  (ROP)  includes  a 
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assessment  and 
our  own,  we  took 
the  opportunity  to 
talk  with  that 
student . 


student's  self  evaluation/ teacher  evalua- 
tion and  a discussion  period  for  the 
student  and  teacher  to  talk  about  their 
assessments  and  possible  next  steps. 

In  the  first  weeks  of  using  the  record 
with  my  (Loretta)  class,  I found  that  most 
of  the  students  tended  to  give  them- 
selves lower  ratings  than  I gave  them. 
This  not  entirely  unexpected  result  gave 
rise  to  a useful  class  discussion  about 
low  self  esteem.  As  the  weeks  pro- 
gressed, however,  and  the  class  became 
accustomed  to  the  weekly  routine  of 
reflection  and  assessment,  I found  that 
we  usually  concurred.  Whenever  notice- 
able discrepancies  occurred  between  a 
student  assessment  and  our  own,  we 
took  the  opportunity  to  talk  with  that 
student . 

A student  believed  she  deserved  a '3' 
in  her  math  area.  She  believed  that  she 
accomplished  obtaining  and  using 
information.  I had  rated  hera'l.' 
because  she  had  not  advanced  according 
to  her  goal  plan.  Her  success  was  a 
repeat  of  the  preceding  months  and  she 
had  not  advanced  since  then.  A discus- 
sion of  her  achievement  and  an  explana- 
tion followed.  The  student  revealed  she 
did  not  want  to  leave  the  comfort  zone 
she  was  in  and  move  into  algebra.  She 
was  encouraged  to  address  her  fears,  as 
it  was  on  her  goal  plan.  She  was  also 
assured  of  support  and  help  with  her 
move.  A discussion  about  her  learning 
style  followed  and  a plan  was  put  into 
place  for  her  to  approach  the  new  work. 

Another  student  assessed  her  atten- 
dance one  month  asa'3' , when,  in  fact, 
she  had  missed  half  of  the  class  time 
through  absences  and  tardiness.  She 
was  dismayed  to  see  that  I had  assessed 


her  attendance  at  'V  .We  sat  down  and 
discussed  the  issue,  both  presenting  our 
points  of  view.  In  subsequent  weeks  her 
punctuality  improved  but  she  stiU 
missed  many  classes.  She  maintained 
her  self  assessment  at  3 while  my 
assessment  of  her  on  this  particular 
issue  rose  to  2. 

The  Record  of  Participation  proved  to 
be  very  useful  as  an  ongoing  assessment 
tooL  It  was  very  helpful  in  keeping 
students  focussed  on  their  goals  and 
assuring  accountability  and  progress, 
when  used  in  conjunction  with  a learn- 
ing contract  completed  at  the  beginning 
of  cycle.  We  had  a very  short  time  to  test 
out  our  tool;  students  would  benefit 
from  using  the  tool  for  a year.  It  will  be 
exciting  to  spend  the  year  using  the  tools 
and  comparing  previous  attempts  at 
assessment  and  refine  our  tool  so  that  it 
better  meets  the  needs  of  the  students. 

CONCLUSION 

The  process  of  mentoring  gave  us  an 
opportunity  to  plan  and  experiment 
with  tools  and  ideas  that  we  would  have 
not  normally  had  the  time  or  support  to 
do.  It  also  enabled  us  to  come  up  with 
the  most  complete,  accurate  and  efficient 
way  to  access  the  whole  student  as  a 
person  who  is  developing  in  a new 
environment  and  to  look  more  closely  at 
those  factors  that  would  affect  his  or  her 
full  participation  in  the  class  or  towards 
attainment  of  a goal  The  process  also 
gave  us  an  opportunity  to  work  together 
as  professionals,  sharing  ideas  and 
issues  and  using  our  experiences  and 
expertise  to  solve  critical  issues  that  face 
our  students  in  the  classroom. 
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Flail  RK  1 

Record  of  Participation 


Name: 


I -least  2-middle  3- most 


Days/Hours  present 
Show  of  commitment 
Willingness  to  try  new  activities 


a I'-L.  i 4i  el_*n_ 

rv^juinpiidii  rvtum 


Independent  work 
Group  work 

Complete  homework  assignments 
Accomplish  Writing  Skills 


Work  completed  on  time 
Class  participation 
Extra  work  completed 
Accomplish  Social  Studies  Skills 


Accomplish  Job  Skills 
Progress  towards  long  term  goals 


2. 

3. 


1. 

2. 

3. 


1. 

2. 

3. 


WK 

1 

ST 


COMMENTS: 


WK 

2 

ST 


WK 

3 

ST 


WK 

4 

ST 


WK 

5 

ST 


WK 

6 

ST 


££-£■ 
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Working  with  Parents 

Authentic  Assessment  in  Family  literacy  Programs 


by 

Pauline  O'Leary 
Barbara  Krol- 
Sinclair 


Even  Start  Family 
Literacy  Program 
Dorchester,  MA 

& 

Intergenerati  onal 
Literacy  Project 
Chelsea,  MA 


During  the  Partnership  Project,  Barbara 
Krol-SincUdr  and  Pauline  O'Leary  worked 
together  to  develop  a tool  that  would  help 
parents  set  goals  with  their  children  and 
measure  the  achievement  of  those  goals.  It 
was  a way  to  encourage  parents  to  take  an 
active  role  in  planning  and  evaluating  their 
literacy  practices  with  their  children. 

Both  of  us  work  in  family  literacy 
programs  operating  in  the 
Boston  area.  Pauline's  program  is 
an  Even  Start  project  in  its  third  year  and 
is  located  at  the  ABCD  building  on 
Geneva  Avenue  in  Dorchester.  The  pro- 
gram is  targeted  at  Head  Start  parents 
and  their  children.  Together,  Even  Start 
parents  and  staff  work  to  create  an 
environment  which  nurtures  self- 
confidence.  In  the  program,  parents 
focus  on  their  own  reading,  writing, 
math  and  English  skills.  They  gain  an 
awareness  and  understanding  of  their 
children's  learning  and  how  to  continue 
the  family's  education  at  home.  Two 
classes  are  offered,  one  for  parents 
whose  first  language  is  English,  the 
other  for  families  for  whom  English  is  a 
second  language.  A total  of  40  families 
are  served  by  the  program. 

Barbara's  program,  the  Intergenera- 
tional  Literacy  Project  in  Chelsea  — in  its 
sixth  year  of  operation  — is  a collabora- 
tive effort  of  the  Chelsea  Public  Schools 
and  Boston  University  School  of  Educa- 
tion. The  program  is  designed  to  help 
parents  and  other  adult  family  members 


strengthen  their  own  literacy  and 
enhance  the  strategies  they  use  in 
literacy  interactions  with  their  children. 
Flexible  grouping  and  cooperative 
learning  are  emphasized  in  instruction, 
and  materials  include  a range  of  articles 
of  adult  interest,  brochures  on  family 
literacy  and  children's  storybooks.  Four 
multi-level,  multi-lingual  classes  are 
offered,  serving  100  families,  for  most  of 
whom  English  is  a second  language. 

In  beginning  work  as  a mentoring 
team,  we  had  an  advantage  over  pairs 
who  were  unfamiliar  with  each  other's 
work.  First  of  all,  we  already  knew  each 
other,  we  had  met  several  times  over  the 
past  few  years  at  sharing  meetings  for 
family  literacy  programs  at  the  Adult 
Literacy  Resource  Institute  and  have 
some  familiarity  with  each  other's  pro- 
grams and  the  issues  we  each  face,  both 
practical  (such  as  space  limitations)  and 
philosophical  (including  the  ongoing 
question  of  how  best  to  meet  the  diverse 
needs  and  interests  of  learners). 

Our  second  bond  was  that  Pauline 
had  visited  Barbara's  program  two  years 
before  and  had  an  understanding  of  how 
that  program  operates. 

Most  importantly,  both  programs 
focus  on  promoting  family  literacy. 
While  there  are  some  very  different 
approaches  to  intergenerational  literacy 
across  many  programs,  the  two  of  us  are 
fortunate  to  share  a common  view  of  the 
families  with  whom  we  work,  the  role  of 
our  programs  and  of  effective 
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approaches  for  facilitating  literacy 
practices  within  families.  We  both 
believe  that  family  literacy  programs  can 
best  meet  the  needs  of  participating 
families  by  building  on  the  literacy 
interactions  already  in  place  in  the 
home. 

The  result  is  that  we  were  both 
delighted  to  have  the  opportunity  to 
work  together  on  this  project. 

THI  INITIAL  MUTINO 

Before  the  entire  group  of  partners 
met  in  January,  we  decided  to  have 
Barbara  visit  Pauline's  program.  After 
class  ended,  we  sat  down  to  discuss  our 
programs,  including  their  objectives, 
schedules,  the  activities  emphasized  — 
and  assessment. 

For  both  of  our  family  literacy 
programs,  one  of  the  primary  functions 
of  assessment  tools  is  to  increase 
parents'  metacognitive  awareness  of  the 
literacy  activities  they're  engaging  in 
with  their  children  on  a regular  basis, 
that  is,  to  help  parents  learn  to  value 
what  they're  already  doing  with  their 
children,  and  to  expand  the  range  of 
strategies  and  activities  they  use  in 
interacting  with  their  children. 

We  determined  that  whatever 
assessment  tool  we  came  up  with,  it 
must  serve  to  inform  parents  about  their 
strengths  and  needs  in  developing 
literacy  activities  in  which  to  engage 
with  their  children,  in  addition  to 
demonstrating  change  to  the  teacher  and 
the  program. 

OOAL-SCTTINO 

Pauline's  program  is  two-pronged, 
directed  toward  both  helping  parents 
focus  on  their  own  literacy  development 
and  working  with  parents  to  aid  them  in 


planning  and  carrying  out  literacy 
activities  with  their  children.  During  the 
time  parents  are  on-site,  they  take  part  in 
a literacy  class,  work  with  their  teacher 
in  planning  for  the  Parent  and  Child 
Together  time  (P.A.C.T.),  and  the 
P.A.C.T.  session  itself,  a block  of  time  in 
which  parents  engage  in  activities  with 
their  children  that  encourage  a sense  of 
discovery  and  play  and  improve  parent- 
child  communication. 

We  decided  to  limit  the  scope  of  the 
project  at  this  point  to  developing  a tool 
that  could  be  used  by  parents  in 

planning  f nr  anH  acggsging  fhpir 

activities  with  their  children  during  the 
P.A.C.T.  Pauline  was  interested  in 
helping  parents  target  their  goals  for  a 
session  and  to  determine  how  they 
would  accomplish  these  objectives.  She 
especially  wanted  parents  to  develop 
skills  necessary  to  evaluate  the  success 
of  the  sessions  afterwards.  Barbara's 
experience  in  family  literacy  has  been 
limited  to  her  own  program,  so  she  was 
excited  about  the  prospect  of  adapting 
ideas  she  had  had  success  with  to 
another  program  format. 

INITIAL  PLANNINO 

Our  next  meeting,  also  held  at 
Pauline's  program,  was  devoted  to 
brainstorming.  Pauline  discussed  the 
P.A.C.T.  sessions,  how  they  were 
conducted  and  the  assessment  that  was 
currently  in  place  (see  Figure  1).  One  of 
her  concerns  was  that,  although  parents 
were  actively  planning  for  their  sessions 
with  their  children,  after  the  P.A.C.T. 
they  were  not  using  what  they  had 
learned  from  the  session  in  planning  for 
future  P.A.C.T.S.  Parents  didn't  seem 
connected  enough  to  the  activities  or 
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confident  enough  in  their  own  abilities 
to  be  able  to  explain  what  had  been 
effective  and  what  they  needed  to  work 
on  next. 

We  tossed  around  ideas  for 
developing  a form  that  would  allow 
parents  to  make  the  connections  between 
the  activities  that  they  were  engaging  in 
in  the  program  (in  the  P.A.C.T.  session) 
and  at  home,  to  begin  to  recognize  a 
continuum  in  their  children's  learning, 
connectedness  between  individual 
literacy  activities  and  P.A.C.T.  sessions. 

UFININO 

In  our  next  meeting,  we  determined 
the  exact  design  of  the  form  (see  Figure  3) 
building  on  one  already  in  use  (Figure  2). 
Our  previous  session  had  consisted  of 
the  two  of  us  tossing  out  ideas  and 
building  on  them.  This  time,  we  sat 
down  and  wrote  out  how  we  wanted  the 
form  to  look  and  focused  on  the  exact 
wording  we  felt  would  best  help  parents 
in  assessing  what  they  were  doing  with 
their  children. 

We  assigned  outselves  the  task  of 
focusing  specifically  on  the  form 
previously  in  use,  what  we  wanted  to 
know,  and  how  it  could  be  changed  to 
give  more  useful  information  to  the 
parents,  the  teacher  and  the  program. 
Both  of  us  seemed  to  feel  that  our 
intuitive  brilliance  of  the  last  meeting 
had  somehow  slipped  away,  but  we 
came  up  with  a workable  tool 

TSSTINO  OUT 

Because  of  problems  in  getting 
together,  Pauline  was  only  able  to  pilot 
the  resulting  assessment  tool  for  three 
weeks,  not  ideal  but  enough  to  give  us 
an  idea  of  its  strengths  and  weaknesses. 
We  were  unable,  given  the  time 


constraints,  to  meet  with  participating 
parents  to  determine  their  views  on  the 
two  forms  and  how  they  might  help 
parents  in  their  literacy  interactions  with 
their  children. 

Pauline  found  that  the  new  assess- 
ment tool  was  disappointingly  similar  to 
the  form  she  had  previously  been  using, 
in  that  parents  were  completing  it  in  the 
same  way.  With  both  forms,  parents 
were  putting  most  of  their  effort  into 
their  planning  for  the  day's  session 
(before  P.AC.T.).  In  describing  "What  I 
did"  and  "How  I did  it",  on  both  the  old 
and  new  tools  parents  were  writing 
answers  of  only  one  or  a few  words. 

The  new  format  had  not  provided 
parents  with  a clearer  way  in  which  to 
describe  their  interactions  with  their 
child  or  to  analyze  the  strategies  they 
had  used  in  the  activity.  Pauline  felt  that 
possibly  parents  still  had  difficulty  in 
reporting  what  they  had  done  after  the 
session  because  they  had  already 
described  their  activities  in  writing 
before  the  P.AC.T.  session;  they  saw  no 
need  to  duplicate  their  efforts. 

There  were  some  significant  areas  of 
difference  between  the  old  and  new 
tools,  however.  Our  new  form  had 
added  the  phrase,  "this  week  I wanted  to 
help  my  child.."  Pauline  felt  that  this 
prompt  helped  parents  to  focus  and 
report  their  objectives  more  clearly.  In 
addition,  the  new  tool  provided  a more 
structured  framework  and  a better 
layout,  and  employed  language  that  the 
parents  were  more  likely  to  use.  Most 
importantly,  in  using  the  new  assess- 
ment tool  parents  were  focused  for  the 
first  time  on  setting  goals  for  themselves 
and  their  children  in  their  next  P.A.C.T. 
session. 
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On  the  original  form  used  by  parents 
after  P.A.C.T.,  all  wording  was  in  second 
person,  Le.  "you".  In  the  tool  we  created, 
we  changed  the  wording  to  first  person, 
"I".  We  hoped  that  this  change  would 
empower  the  parents  who  were  analyz- 
ing their  P.  AC.T.  sessions,  to  give  them 
the  sense  that  they  were  in  control  of 
their  interactions  with  their  children. 

In  setting  agendas  for  the  future,  we 
hope  that  we  can  use  this  new  tool  to 
assist  parents  in  carrying  their  self 
assessment  of  their  activities  with  their 
children  into  their  daily  routines  at 

Krtmo  TAT/*  Knll  ATTA  fk <5 f fVlA  nAUT  IaaI 
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is  better  suited  to  such  an  extension 
because  the  parent  is,  in  essence, 
analyzing  her  activities  for  her  own 
reflection  and  use  in  setting  future 
agendas;  even  though  program  staff  also 
has  access  to  the  form,  the  parent  is 
writing  for  her  own  authentic  use  and 
not  merely  as  a report  to  the  outside 
authority. 

We  also  feel  that  our  new  tool  helps 
parents  to  better  link  their  individual 
sessions  — that  we  have  succeeded  in 
building  awareness  of  a learning 
continuum  since  now,  at  the  end  of  one 
session,  parents  are  focused  on  what 
they  might  want  to  do  the  next  time. 

CONCLUSIONS 

One  tool — one  piece  of  paper — may 
not  seem  like  a substantial  outcome  from 
half  a year's  worth  of  meetings  and 
phone  calls.  We're  tempted  to  become 
defensive,  but  it's  clear  to  us  that  the 
nature  of  collaboration  requires  a great 
deal  of  groundwork.  We  could  easily 
have  made  minor  adaptations  to  tools 
already  in  use  in  Barbara's  program.  Our 


goal,  however,  was  to  begin  to  build  a 
new  model  for  parents  to  use  in 
assessing  their  literacy  interactions  with 
their  children.  In  that,  we  feel  that  we've 
succeeded.  In  addition,  we  have  engaged 
in  a great  deal  of  reflection  on  the 
processes  we  engage  in  with  our 
participating  families  on  a regular  basis 
and  the  role  of  parents  in  setting  their 
own  literacy  agendas.  Such  insights  help 
us  to  continually  assess  our  own 
behaviors  and  to  revise  our  interactions 
with  families. 

NUT  STEPS 

Even  though  the  project  has  officially 
ended,  we  intend  to  continue  working 
together  in  the  fall.  We  would  like  to 
make  further  refinements  to  the  tool 
we've  tested,  as  well  as  look  for 
additional  ways  to  help  parents  assess 
and  reflect  on  their  literacy  interactions 
with  their  children  and  their  own 
literacy  development.  Through 
interviews  with  participating  parents, 
we  would  like  to  get  their  input  into 
ways  to  help  them  assess  the  strategies 
they  use  in  their  interactions  with  their 
children.  In  addition,  we'd  like  to 
develop  is  a method  to  help  parents 
systematically  monitor  their  literacy 
activities  at  home.  For  both  of  us,  the 
goals  are  to  validate  the  literacy 
activities  in  which  families  engage  in  the 
home,  to  help  parents  systematize  the 
process  of  assessing  their  literacy 
interactions  with  their  children  and  to 
inevitably  make  our  formalized 
programs  of  reflection  family  literacy 
redundant  for  participating  families.  We 
have  only  begun  the  process. 

or 
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FIOURI  1 

Parent  and  Child  Time  (PACT) 


BEFORE: 


Decide  WHAT  you  wont  to  leach 
Plan  how  you  will  teach  it 
find  materials  that  will  help  you 

DURING 

Introduce  the  materials  to  your  child 

Listen  to  your  child 

Respond  to  your  child's  questions 

Use  words  that  affirm  your  child's  efforts 

Review  or  summarize  the  activity  before  you  put  it  away 

AFTER 

Review  haw  you  foe! 

Record  haw  your  child  did  with  the  activity 
Write  a journal  entry  about  the  lesson 
Decide  how  you  will  follow  up  at  home 
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noun  a 

PACT 

EVEN  START  PACT  Preparation 

Date: Name:. 

1 .  What  I want  to  teach: 


2.  Materials  I need: 


3.  How  I will  teach  it: 
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FIGURE  3 

PACT 


BEFORE  PACT: 

This  week  I wanted  to  help  my  child: 


Strategies: 


I want  to  remember  to: 


AFTER  PACT 
What  I did: 


How  I did  it: 


Next  time,  we  will  work  on: 


r o n 
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Taking  Time  to  Talk: 

Students  and  Teachers  Setting  Goals 


Haverhill  Community  Action's  " Entry 
Employment  Experience  Program ” served 
(until  its  funding  was  cut  at  the  end  of  June, 
1994)  30  at-risk,  drop-out  youths  aged  16  to 
21  each  year.  The  program  was  designed  to 
give  the  participants  necessary  life  and  work 
skills,  help  them  toward  a G.E.D.,  and  assist 
them  in  finding  full-time  employment,  skills 
training  or  higher  education  as  ihey  com- 
pleted the  program.  Marty  Tassi-Richardson, 
a teacher  in  this  program  was  the  mentor  in 
the  Alternative  Assessment  Partnership 
Project.  Marty  has  since  been  hired  to  teach 
in  a similar  program  at  the  Blackstone  Valley 
Regional  Vocational  Technical  School  in 
Upton,  Massachusetts. 

The  Barnstable  County  House  of  Correc- 
tion Adult  Basic  Education  program  offers 
classes  in  ABE,  Pre-ASE,  and  ASE.  The 
institution,  which  was  built  to  house  75 
inmates,  now  houses  approximately 200. 
About  ten  percent  of  the  inmates  are  female. 
At  any  given  time,  about  half  of  the  inmates 
are  school  drop-outs  who  do  not  have  a 
G.E.D.  Inmates  who  have  a high  school 
diploma  or  G.E.D.  are  able  to  attend  classes 
to  brush-up  on  skills.  Classes  are  open- 
ended.  Student  population  is  very  transient. 
Students  are  always  leaving  due  to  comple- 
tion of  sentence,  parole,  or  transfer  to 
another  institution.  New  students  are 
always  arriving.  Classes  also  have  to  compete 
with  the  opportunity  to  work  at  a job  either 
inside  the  institution  (kitchen,  laundry, 
canteen)  or  outside  (county  form,  commu- 
nity service).  Inmates  may  reduce  their 
sentences  by  up  to  six  days  a month  for 
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attending  classes  (2  days),  working  (2  days), 
and  participating  in  Lifeline  or  Inside-Out 
programs  (2  days).  Ddrdre  McLaughlin,  a 
teacher  at  the  Barnstable  County  House  of 
Correction,  is  the  mentee  in  the  partnership 
project. 


y|*  artha  Germanowski , a 
■ ■ teacher  for  Haverhill  CoiTi- 

JL  ▼ JL  munity  Action,  had  devel- 
oped the  Educational  Goals  Assessment 
Package  (EGAP)  for  use  in  her  Step-By- 
Step  program  (an  education  program  for 
the  homeless  in  AmesburyXsee  AiA,  Fall 
1991).  Marty  had  adapted  the  EGAP  for 
use  in  her  youth  program  and  had  been 
using  it  for  over  a year  when  she  became 
involved  in  the  alternative  assessment 
project.  She  felt  that  the  package  could 
be  used  more  effectively  and  the  project 
provided  a structured  way  to  explore 
this  possibility. 

Marty  was  concerned  with  a lack  of 
engaged,  thoughtful  responses  from 
many  participants  on  both  the  initial 
intake  assessment  and  the  weekly  log. 
She  felt  that  if  she  more  clearly  defined 
the  function  of  the  assessment  in  the 
program  for  herself  and  students,  they 
would  respond  in  a more  meaningful 
way.  She  wanted  to  make  the  use  of  the 
assessment  tool  meaningful  and  not  just 
another  "part  of  the  program"  to  be 
gotten  through  as  quickly  as  possible. 

In  the  past,  Deirdre's  intake  involved 
the  student's  filling  out  a form  which 
asked  basic  Questions  necessary  for 
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completing  DOE  statistical  reports: 
name,  age,  date  of  birth,  place  of  birth, 
first  language,  last  grade  completed, 
possession  of  high  school  diploma  or 
equivalency  certificate,  and  ethnicity. 
The  students  were  also  asked  their 
reason  for  wanting  to  attend  education 
classes.  At  the  same  intake  session,  the 
students  were  asked  to  write  a few 
paragraphs  describing  their  last  experi- 
ence in  school.  They  were  asked  to  be 
specific  about  whether  the  experience 
was  a happy  one  or  an  unhappy  one  and 
why.  They  were  also  asked  what  they 
hoped  to  accomplish  in  the  classes. 

After  intake  information  is  completed, 
Deirdre  administers  the  TABE  (Test  of 
Adult  Basic  Education)  to  determine 
initial  grade  levels  in  reading,  math  and 
language.  The  new  student  is  assigned 
to  classes  based  upon  the  TABE  results, 
with  preference  given  to  those  who  do 
not  have  the  diploma  or  G.E.D. 

In  her  program,  Marty  used  the  MAT 
(Metropolitan  Achievement  Test)  to 
measure  reading  and  math  levels.  In  her 
program,  Marty  used  the  MAT  (Metro- 
politan Achievement  Test)  to  measure 
reading  and  math  levels.  Through 
discussions  with  students  and  observa- 
tions, both  engaged  in  informal  double- 
checks on  the  accuracy  of  these  stan- 
dardized assessments  and  the  informa- 
tion they  provided.  Deirdre  asks  the 
student  directly  about  how  accurate  they 
think  the  results  of  the  test  are.  Marty 
used  the  adapted  EGAP  to  do  the  same 
thing. 

Deirdre's  main  focus  on  entering  the 
partnership  project  was  to  integrate 
student  goals  more  formally  into  the 
program.  Student  goals  had  always  been 
discussed,  but  had  never  been  written 


down,  except  in  a very  small  portion  of 
the  general  intake  form.  Ongoing  goals 
review  and  resetting  had  to  be  formal- 
ized. 

OITTINO  TO  KNOW  EACH  OTHER 

We  met  for  the  first  time  over  lunch 
before  the  project's  April  8 meeting  at 
DOE  headquarters,  though  we  had 
spoken  on  the  phone  the  week  before. 

At  the  April  meeting  we  decided  that 
we  should  formalize  our  ongoing  assess- 
ment, making  it  a regular  part  of  classes. 
We  also  wanted  to  rely  more  on  student 
goal-setting  and  regular  review  of  the 
weekly  or  monthly  plans  used  to  achieve 
those  long-term  goals.  We  decided  that 
in  order  to  improve  the  ongoing  assess- 
ment, we  had  to  improve  the  initial 
assessment  process  as  well.  We  had  to 
formalize  our  initial  dialogue  with 
students  on  their  learning  experience 
and  standardized  test  scores.  From  there, 
we  could  create  an  ongoing  dialogue 
which  uses  that  initial  assessment  as  a 
reference  point  to  allow  students  and 
teachers  to  meaningfully  assess  their 
progress.  We  set  dates  to  visit  each 
other's  classrooms  and  agreed  to  read 
through  the  Tool  Kit  in  the  meantime, 
pulling  out  any  relevant  tools  which  we 
would  consider  in  reworking  our  intake. 

After  visiting  each  other's  classrooms, 
we  discovered  that  the  environments 
and  the  populations  we  served  had 
many  similarities.  The  participants  in 
both  programs  shared  common  educa- 
tional backgrounds  and  faced  some  of 
the  same  obstacles.  Most  of  the  partici- 
pants had  experienced  some  degree  of 
academic,  social  and  emotional  difficulty 
in  their  previous  school  experiences.  In 
some  cases  these  were  complicated  by 
drug  and/or  alcohol  abuse  or  physical 
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abuse  at  home  or  in  school.  Poor  atten- 
dance was  a problem  in  both  programs 
for  all  of  the  above  reasons. 

What  differed  were  the  practical  day- 
to-day  procedures.  Deirdre  works  with 
students  of  all  ages.  She  may  see  a given 
student  only  once  a week  for  2 hours,  or 
up  to  three  times  a week,  depending  on 
the  student's  need  to  work  on  reading, 
math,  and/or  language.  Marty's  pro- 
gram met  five  days  a week  for  three 
hours  a day  and  was  limited  to  16  to  21- 
year-olds  . Deirdre  may  have  up  to  80 
students  at  any  given  time,  some  work- 
in?  on  onlv  one  subiect  area,  and  others 
on  several  areas.  Because  their  class 
hours  are  more  limited,  some  of  the 
things  Marty  decided  would  be  appro- 
priate for  her  program  were  not  feasible 
for  Deirdre.  Deirdre's  program  often  had 
to  compete  with  constantly  changing  jail 
schedules  for  work  details,  court  appear- 
ances, medical  appointments,  visits,  etc. 

When  Marty  visited  Deirdre's  pro- 
gram in  April,  she  was  dissatisfied  with 
how  the  weekly  objectives  sheets  she 
had  been  using  with  her  students  had 
been  working.  Students  were  not  consis- 
tent in  filling  them  out  and  didn't  feel 
that  they  were  necessary.  Students  had 
been  setting  objectives  on  Monday,  and 
reviewing  them  on  Friday.  Marty's 
discussion  with  Deirdre  led  to  the 
conclusion  that  weekly  plans  needed  to 
be  an  integral  part  of  each  class  day.  The 
previous  year,  Marty  had  students  set 
and  track  objectives  each  class  day.  This 
had  seemed  to  be  too  much  for  students 
and  had  led  to  the  abbreviated  process 
she  was  using  this  year. 

Our  ongoing  discussions  led  to  the 
development  of  a new  process.  Marty 
will  ask  students  to  set  objectives  at  die 


beginning  of  the  week,  but  will  ask  them 
to  comment  on  their  work  toward  those 
objectives  each  day.  These  weekly  plans 
will  be  filed  along  with  the  initial  intake 
form,  developed  as  a result  of  this 
partnership  project,  for  teacher  and 
student  reference.  Marty  will  also  meet 
monthly  with  each  student  and  the 
program  counselor  after  program  hours. 
For  these  meetings,  the  student  would 
be  given  a copy  of  his/her  file  which 
included  the  initial  intake  form.  The 
teacher  and  the  counselor  would  review 
the  forms  beforehand  in  order  to  focus 
on  the  student's  successes  and  in  order 
to  encourage  further  achievement. 
Students  might  wish  to  modify  their 
original  responses  and  / or  goals. 
Deirdre's  students  will  review  objectives 
on  a monthly  basis,  but  she  will  ask 
students  to  write  a brief  comment  about 
the  list,  what's  been  accomplished, 
problems,  etc.,  each  week.  She  started 
using  the  goal  sheets  with  a pilot  group 
of  ten  students  in  June. 

In  order  to  create  the  new  intake  form, 
we  pulled  items  from  the  Toolkit  and 
combined  them  with  Marty's  adapted 
EGAP.  Some  ideas  from  the  Toolkit  were 
added  verbatim;  others  were  modified  in 
ways  which  we  thought  may  produce 
more  of  a response  from  our  students. 
We  rearranged  the  order  of  questions  on 
the  intake  a number  of  times.  One  major 
change  we  made  to  Marty's  adapted 
EGAP  was  taking  out  the  Career  Goals 
and  Personal  Goals  sections  and  substi- 
tuted it  with  a section  of  questions  that 
would  encourage  students  to  think  more 
creatively  and  in-depth  about  goals  and 
objectives. 

At  the  June  3 project  meeting,  we 
were  both  fine-tuning  what  had  been 


The  teacher  and 
the  counselor 
would  review  the 
forms  beforehand 
in  order  to  focus 
on  the  student's 
successes  and  in 
order  to  encourage 
further 
achievement. 
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produced.  Even  as  we  passed  out 
samples  to  other  mentor-mentee  part- 
ners at  the  meeting,  we  had  to  explain 
that  we  were  going  to  rearrange  certain 
sections. 

We  both  felt  that  working  together 
was  a wonderful  experience.  The  way 
that  the  partnership  project  was  set  up 
enabled  us  to  accomplish  a lot  in  a short 
period  of  time.  Our  meetings  and  con- 
versations gave  us  a chance  to  problem- 


solve  and  brainstorm.  Having  a partner 
often  encouraged  us  to  see  things  from  a 
different  perspective  and  to  get  feedback 
so  that  the  process  of  eliciting  and 
implementing  solutions  was  much  more 
efficient.  The  larger  project  meetings 
gave  us  a further  chance  to  reflect  on  the 
partnership  and  share  ideas.  All  this  in 
turn  enabled  us  to  better  meet  the  needs 
of  our  students,  which  is  our  most 
important  GOAL. 
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FIOUM  1 

Intake:  Barnstable  County  House  of  Corrections 

Full  Name: 

Date: 

Date  of  Birth:  Age: 

Place  of  Birth:  (city) (state)  ~ 

First  Language: 

Last  grade  you  attended: 

Last  grade  you  completed: 

Do  you  have  a high  school  diploma: 

Do  you  have  a G.E.D.?  

Do  you  have  a college  degree?  

Name  and  address  of  the  last  school  you  attented: 


Check  one: 

□ White  - Not  Hispanic  □ American  Indian 

□ White*  Hispanic  □ Asian  - Pacific  Islander 

□ Black*  Not  Hispanic  □ Other 

□ B I ack  * Hispanic 

What  is  the  reason  for  wanting  to  attend  Adult  Education  classes? 


I would  like  your  experience  in  Adult  Education  classes  to  be  pleasant  and  rewarding.  Describe 
below  your  past  experince  in  school.  Tell  me  if  it  was  happy  or  an  unhappy  one.  If  you  did  not 
complete  high  school,  please  tell  me  why  you  left  school.  Was  it  your  choice  to  leave?  Please  tell 
me  also  what  you  hope  to  accomplish  in  these  classes  (receive  a G.E.D.,  brush  up  on  skills,  etc.). 
Be  specific. 
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II.  History:  in  this  section  I would  like  you  to  describe  your  past  experience  with  school.  Was 
your  time  in  school  a pleasant  or  unpleasant  experience?  Did  you  receive  your  diploma  or 
G.E.D.?  If  you  left  school  was  it  your  own  decision  or  were  you  asked  to  leave?  Please  be 
specific. 


III.  Stif-Rating:  This  section  of  the  questionnaire  gives  you  a chance  to  think  about  your  skills 
and  communicate  to  the  teacher  what  you'd  like  to  learn  and  how  you  learn  best.  The  education 
program  here  is  designed  to  help  you  improve  your  skills  and  reach  your  goals.  To  make  it 
work,  you  must  communicate  goals,  strengths  and  needs  as  clearly  as  you  can. 

There  are  no  right  or  wrong  answers.  The  more  information  you  can  give  in  any  answer,  the 
better.  You  may  use  extra  paper  if  you  need  more  space  for  your  answers. x 

My  reading  skills  are:  □ Excellent  □ Good  □ Fair  □ Need  Work 

Do  you  understand  most  of  what  you  read?  


When  do  you  read? 

What  do  you  like  to  read? 


Is  there  anything  that  you  really  dislike  reading? 

My  math  skills  are:  □ Excellent  □ Good  □ Fair  □ Need  Work 
What  do  you  do  best  in  math  and  what  is  difficult  for  you  in  math?  _ 


My  writing  skills  are:  □ Excellent  □ Good  □ Fair  □ Need  Work 
Are  you  a confident  writer?What  do  you  like  to  write? 


Identify  and  describe  at  least  5 personal  achievements,  skills,  or  good  characteristics. 

1. 

2. 

3. 
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4.  

5.  

Do  you  learn  better  by  working  with  others  or  alone?  Please  explain. 


Do  you  learn  best  by  watching  someone  else,  by  reading  about  something,  by  listening  to 
someone,  or  by  doing  something  yourself?  Please  give  an  example  of  something  you  learned 
well  and  how  you  learned  it. 


If  you  do  not  understand  a word  when  you  are  reading,  do  you  look  at  the  other  words  around 
it  and  try  to  figure  out  whaf  ii  means  (in  context}?  When  have  you  done  this?  


If  you  do  not  understand  a word  when  you  are  reading,  do  you  look  it  up  in  the  dictionary? 
When  have  you  done  this?  ] 


My  favorite  subject  in  school  was 


because 


The  most  memorable  thing  I ever  learned  in  school  was 


Describe  a situation  where  you  were  able  to  accomplish  something  that  you  did  not  think  you 
could  do,  or  where  you  helped  someone  else  to  accomplish  something,  (examples:  solving  a 
problem  between  people,  doing  a budget,  doing  something  good  for  a family  member  or  friend, 
doing  something  good  for  your  health.) 


Have  you  ever  been  unable  to  do  something  because  you  didn't  have  the  necessary  skills  or 
information?  Please  describe.  
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IV.  Goals:  In  this  section  you  should  describe  what  you  hope  to  accomplish  in  this  program. 
Please  be  specific. 

In  one  year  I hope  to 


I wish  I could 


List  two  things  that  you  have  seen  or  heard  about  and  you  would  like  to  understand  better. 

1. 

2. 

List  at  least  5 things  that  you  are  doing  to  get  from  here  to  there: 

1. 

2. 

3. 

4.. 

5. 

Other  things  I would  like  to  learn:  (Please  describe  any  experience  you  have  with  the  following.) 

managing  money  effectively _ 

understanding  pricing,  discounts,  etc.  ___ 

registering  to  vote 

getting  a library  card 

learning  how  to  use  the  library 

getting  involved  in  community  activities 

learning  about  careers  such  as 

protecting  my  health  effectively 

managing  time  effectively 

other 
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V.  Interests 

In  my  free  time  I like  to 


I have  written  or  would  like  to  write: 

□ letters  to  school  □ papers  □ advice  to  others 

□ stories  □ news  articles  □ ioumal/diary 

□ songs  □ poems  □ essays 

I like  to  read  the  following  for  enjoyment.  Please  list  the  title  of  anything  you  have  read  recently 
in  any  category 

adventure 

mystery 

romance 

history 

sports : 

music 

poetry 

religion 

horror  

TV  and  movies 

true  stories 

self-help  . . 

how  to 

hobbies 

parenting  

science  fiction 

magazines 

newspapers  
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Monthly  Plan 

Name: 

Date: 

My  goals  for  this  month  include  the  following:  (Include  page  numbers,  specific  tests,  writing, 
math  skills,  as  well  as  understanding  concepts  and  ideas,  etc..} 

1. 

2. 

3.  

4.  

5.  

Please  feel  free  to  make  changes  in  your  goals  during  the  month  and  voice  any  concerns, 
questions  in  the  space  below. 

Date  Student  Comment 


Please  use  the  back  of  this  paper  if  you  need  more  room. 
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FIGURE  3 

Intake 

Entry  Employment  Experience  Program 
Education  Component:  Initial  Interview 

Name: Date: 

Age: First  Language: 

Educational  Backqround: 

Last  school  attended: 

Last  grade  attended: Last  grade  completed: 

Date  last  attended:  

This  questionnaire  gives  you  a chance  think  about  and  communicate  to  the  teacher  what  you'd  like 
to  learn  and  how  you  learn  best.  The  EEE  program  is  designed  to  help  you  learn  and  reach  your  goals. 
To  make  it  work,  you  must  communicate  goals,  strengths  and  needs  as  specifically  as  you  can. 

There  are  no  right  or  wrong  answers.  The  more  information  you  can  give  about  any  answer,  the 
better.  Feel  free  to  use  extra  paper  if  you  need  more  room  to  write. 

SELF-RATING 

My  reading  skills  are:  □ Excellent  □ Good  □ Fair  □ Need  Work 
Please  explain: 

Do  you  understand  most  of  what  you  read? 


When  do  you  read? 


What  do  you  like  to  read?  

If  you  don't  understand  a word  when  you  read,  do  you  try  to  guess  the  word's  meaning  by  looking 
at  the  words  around  it  and  fitting  it  into  the  story?  Can  you  think  of  a time  when  you  did  this? 
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PERSONAL  READING  AND  WRITING 

I like  ta  read  for  enjoyment  (please  list  anything  you've  read  recently  in  any  category): 
adventure 

mystery 

romance 

history 

sports 

music 

poetry 

religion 

horror 

TV  & movies 

true-life  stories 

self-help 

how-to 

hobbies 

parenting  

science  fiction 

magazines 

newspapers  

I have  written  or  would  like  to  write: 

□ poems  □ songs  □ stories  □ joumal/diary  □ essays  □ school  papers 

□ news  articles  □ letters  □ advice  to  others 

Other  reading  or  writing  skills  you  have  or  would  like  to  have: 


The  best  thing  I ever  read  was 

The  best  movie  or  TV  program  I ever  saw  was  . 

II  --  335 
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I also  enjoy: 

□ cartooning  □ painting  □ drawing  □ singing  □ Other 

LIFE  SKILLS 

Describe  a situation  where  you  were  able  to  help  yourself  or  someone  else  accomplish 
something  (some  examples  may  include  solving  a problem  between  people,  doing  a budget, 
doing  something  good  for  your  health,  etc.):  


Have  you  ever  been  unable  to  do  something  you  really  wanted  to  do  because  you  didn't  have 
the  necessary  skills  or  information?  Please  describe 

A personal  characteristic  1 would  like  to  work  on  is 

because 

7 

Name  at  least  two  things  you've  seen  or  heard  about  and  would  like  to  understand  better 
1. 

2. 

Would  you  rather  read  a aood  storv  or  a news  maaazine?  Whv? 

Would  you  rather  listen  to  an  older  relative  talk  about  your  family  history  or  listen  to  the  news?  Why? 

Would  you  rather  discuss  personal  and  social  values  and  how  they  fit  into  your  life  or  how  the  body 
works?  Whv? 

Name  at  least  two  other  things  you  enjoy  reading  about  or  discussing  (some  examples  might  be 
hobbies,  places  you've  lived,  sports,  current  events  or  news  you're  concerned  about,  health,  movies, 
music,  etc.): 

; ••  ~ Ml 
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Do  you  learn  better  working  with  others  or  alone?  Please  explain. 


Do  you  learn  best  by  watching  someone  else,  by  reading  about  something,  by  listening  to 
someone  or  by  doing  something  yourself? 


Please  give  an  example  of  something  you  learned  well  and  how  you  learned  it. 


Identify  and  describe  at  least  5 personal  achievements,  skills  or  good  characteristics: 

1. 

2. 

3.  

4.  

5.  

Please  list  at  least  one  thing  which  you  can  teach  others 


GOALS 

In  one  year  I hope  to  be 
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Think  of  at  least  5 things  you  are  doing  to  get  from  here  to  there: 

1. 

2. 

3.  

4. _ 

5.  

Other  things  I would  like  to  learn  (please  describe  any  experience  with  the  following): 

managing  money  effectively 

understanding  pricing,  discounts,  etc.  

registering  to  vote 

getting  a library  card  

learning  to  use  the  library 

getting  involved  in  community  activities 

learning  about  careers  such  as 

protecting  my  health  effectively 

managing  time  effectively 

Please  add  at  least  one  other  thing  you  would  like  to  learn 
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WEEKLY  PLAN 

Name: 

Data: 

My  plan  for  this  week  is  (include  page  numbers,  specific  tests,  writing,  as  well  as  understanding 
concepts  and  ideas,  etc.): 

1. 

2. 

3. 

4. 

5. 

Please  write  down  the  date  each  day  you  are  in  class  and  comment  on  what  you  accomplished 
and  how  class  went  that  day.  Feel  free  to  change  or  add  to  your  list  above  as  the  week  goes  on. 

USE  THE  BACK  OF  THIS  PAGE  IF  YOU  RUN  OUT  OF  ROOM 

Day  Your  Comments  Teacher  Comments 

5 4 4 
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Thoughts  on  Assessment 


The  Log  School  English  as  a 

Second  Language  Program  and 
the  Haitian  Multi  Service  Center 
in  Boston  have  been  working  closely  as  a 
partnership  to  provide  support  services 
for  families  in  Greater  Boston.  One  of 
our  objectives  is  to  help  immigrants 
make  a smooth  transition  from  their 
homelands  to  the  United  States.  Another 
is  to  help  them  achieve  their  goals  and  to 
recognize  this  achievement  by  support- 
ing them  and  teaching  them  the  multiple 
skills  needed  to  function  in  this  country. 

After  attending  last  year's  SABES- 
sponsored  workshops  on  alternative 
assessment,  we  decided  to  explore  the 
issues  further  and  participated  in  The 
Partnership  Project  We  saw  this  project 
as  a way  to  look  at  existing  assessment 
tools  being  used  in  programs  similar  to 
ours  in  the  state. 

We  worked  together  to  analyze  and 
adapt  the  tools  in  the  Toolkit  and  would 
like  to  share  some  thoughts  and  experi- 
ences with  other  members  who  partici- 
pated in  the  project  We  spent  six 
months  in  this  project,  not  developing 
tools  but  instead  understanding  the  tools 
and  experimenting  with  one  to  see  if  it 
would  meet  the  needs  of  our  program 
and  the  students. 

Initial  intake  and  ongoing  assessment 
procedures  can  sometimes  be  difficult 
for  us  as  practitioners  to  understand. 
That  was  the  case  for  us,  too,  although 
we  have  been  assessing  adult  learners 
for  a while.  We  had  to  first  think  about . 


how  our  students  learn  and  how  they 
progress  before  we  could  evaluate  and 
experiment  with  the  tools  in  the  kit 

We  struggled  from  the  start  with  the 
idea  of  alternative  assessment  and  felt  at 
first  that  we  could  not  keep  up  with  the 
dialogue.  It  took  us  a while  and  a lot  of 
effort  to  come  up  with  some  practical 
explanation  of  now  our  learners  learned 
and  progressed  and  what  tools  would 
complement  these  styles.  It  was  also 
difficult  to  measure  students'  learning 
capacity,  especially  adult  immigrants 
whose  goals  and  dreams  have  to  do  with 
financial  independence.  We  saw  one  of 
our  roles  to  be  to  make  sure  they  have 
achieved  academically. 

We  decided  to  try  the  monthly  learn- 
ing form  to  monitor  our  students' 
progress  on  an  ongoing  basis.  We  ob- 
served that  they  were  a little  intimidated 
by  the  length  of  the  forms.  They  even 
said  that  the  forms  we  showed  them 
were  too  much  to  fill  out  every  month. 
We  had  hoped  that  they  would  see  the 
usefulness  of  monitoring  progress 
monthly.  The  students  who  were  not 
capable  of  really  writing  or  reading  were 
reluctant  to  spend  so  much  time  on  the 
forms.  The  ones  that  could  already  read 
and  write  were  more  apt  to  agree  to 
participate. 

We  chose  some  things  from  the 
Toolkit  after  much  discussion.  The  long 
charts  and  logs  easily  scare  students  off 
regardless  of  their  educational  back- 
ground. We  found,  however,  that  the 
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process  of  doing  alternative  assessment  - 
in  this  case,  monthly  learning  forms  - 
took  time  to  be  done  and  time  to  be 
accepted.  We  could  not  assess  these 
learners  in  one  day  and  needed  a process 
that  would  look  at  their  progress  on  an 
ongoing  basis. 

People,  including  instructors,  needed 
to  be  patient  with  each  other  during  the 
experimenting  time.  These  forms  also 
became  part  of  the  teachers'  lesson 
plans,  and  alert  teachers  when  things  are 
not  going  well. 

The  Toolkit  presented  by  The  Partner- 
ship Project  became  an  inspiration  for  all 


mentors  and  mentees.  The  GOALS 
project  staff  members  have  been  won- 
derful in  offering  us  resources  and  help. 
In  sum,  this  project  played  an  important 
role  in  the  area  of  evaluation  and  our 
understanding  of  alternative  assessment 
and  forced  us  to  ask  a lot  of  questions 
about  our  students'  learning  and 
progress.  The  monthly  tool  with  which 
we  chose  to  experiment  had  a tremen- 
dous effect  of  the  students  who  were 
learning  to  become  more  confident  and 
independent  and  also  in  letting  the 
teachers  have  a tool  to  monitor  progress 
monthly. 
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Self  Assessment  for  the  Beginner: 
A Goals  Oriented  Approach 


During  the  1993-4  academic  year 
I had  the  opportunity  to  ex- 
plore self-assessment  tech- 
niques as  part  of  the  Partnership  Project 
in  cooperation  with  the  Massachusetts 
Department  of  Education.  My  partner  in 
this  project  was  Dan  Wilson,  an  ESL 
instructor  at  the  North  End  Union  in 
Boston,  MA.  Although  we  worked  on 
modifying/ developing  our  own  assess- 
ment pieces  we  learned  through  work- 
ing together  that  we  were  facing  similar 
obstacles  in  assessing  the  beginning 
learner  in  a non-threatening  meaningful 
way. 

I grew  up  in  Thailand  and  attended 
university  there.  Every  time  I took  an 
exam,  a question  was  raised  in  my  mind: 
why  must  the  test  format  be  multiple 
choice?  I preferred  exams  that  asked 
questions  and  allowed  me  to  express  my 
opinions  freely.  I felt  that  I could  present 
myself  better  that  way,  whether  or  not  I 
learned  something. 

Now  I am  an  ESL  teacher  in  the 
United  States  at  the  East  Boston 
Harborside  Community  Center.  What  I 
could  not  control  as  a student  I can 
provide  as  a teacher.  My  students  have 
very  limited  English  communication 
skills.  The  level  in  class  is  varied  and 
some  students  have  no  schooling  at  all, 
while  others  have  finished  university  in 
their  countries.  The  unschooled  students 
tend  to  come  from  rural  farming  back- 
grounds and  experience  a number  of 
obstacles.  First  is  the  obvious  problem  of 


beginning  the  educational  process  as  an 
adult  in  a foreign  country.  But  even 
more  difficult  is  the  element  of  fear  Can 
I survive  in  this  country?  Can  I handle 
classroom  learning?  Will  I be  able  to 
adjust  to  a very  different  culture?  Will  I 
ever  understand  the  language  and 
American  customs? 

These  are  just  a few  of  the  questions 
that  can  create  a real  fear  in  the  minds 
and  hearts  of  the  ESL  student 

As  one  can  see,  I was  faced  with  a 
challenging  task  of  instructing  and 
assessing  students  of  very  diverse 
educational  backgrounds  who  may  also 
be  dealing  with  intense  fear  and  anxiety. 
How  could  I give  them  the  same  test? 

I have  been  involved  in  Alternative 
Assessment  for  a few  years.  I was  look- 
ing then  for  a tool  that  was  less  aca- 
demic, related  more  to  my  student's 
lives  and  which  would  be  non-threaten- 
ing to  beginning  learners.  We  originally 
used  the  BEST  test  for  our  program  but 
students  did  not  answer  all  the  questions 
for  a number  of  reasons.  I wanted  to 
create  something  that  was  more  relevant 
to  their  lives  outside  of  class,  like  going 
shopping  etc.  I found  that  I was  testing  a 
lot  of  tools  and  was  very  excited  to  be 
using  alternative  assessment  in  my  class. 
I also  found  that  students  needed  to 
participate  in  these  alternatives.  They 
needed  to  have  a good  attitude  towards 
participating  and  to  be  able  to  see  the 
benefits  of  their  assessment  practices. 

By  trying  out  alternative  assessment  I 
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I learned  that  I 
can  never  use  the 
same  tool  twice. 

I always  have  to 
adapt  the  tools 
according  to  the 
needs  of  each 
group. 


gained  insight  into  my  own  teaching. 

I learned  that  I can  never  use  the  same 
tool  twice.  I always  have  to  adapt  the 
tools  according  to  the  needs  of  each 
group. 

thb  nummsHip 

I first  met  Dan  in  the  Fall  We  talked 
at  length  about  our  students  and  the 
obstacles  we  both  faced  in  assessing 
their  progress.  Dan  also  teaches  begin- 
ning learners  with  limited  communica- 
tion skills.  He  described  his  students  as 
having  the  same  fears  and  the  same 
variety  of  educational  backgrounds.  He 
also  felt  that  current  assessment  tech- 
niques were  not  appropriate  for  all  the 
students  in  his  classes  and  he  had  been 
using  no  system  of  student-driven 
assessment  in  his  class. 

The  challenge  before  us  was  to  help 
him  develop  a system  which  would 
achieve  the  following: 

1) a  student  driven  subjective  self  assess- 
ment; 

2)  a format  that  could  be  handled  by 
those  students  with  very  limitpd 
communication  skills  in  English. 

3)  a format  that  could  be  handled  by 
students  with  little  or  no  education, 
Le.,  some  that  does  not  require  a great 
deal  of  writing  but  which  produces  a 
document  that  can  be  kept  for  review 
and  evidence  of  progress 

4)  a format  that  would  be  appropriately 
mature  for  all  students. 

5)  a format  that  the  students  would  find 
non-threatening 

Dan  and  I shared  some  of  the  tools 
that  I had  been  using  in  my  class.  I had 
been  collecting  student  writing  and 


binding  the  assignments  together  in 
chronological  order.  In  this  way  I was 
creating  a student  journal  of  their  writ- 
ings. Dan  liked  the  idea  of  giving  the 
students  frequent  short  writing  assign- 
ments and  using  these  to  gauge  the 
student's  progress  over  time.  Dan 
thought  this  approach  was  effective  and 
also  useful  because  it  met  all  the  require- 
ments above.  Dan  implemented  this  in 
his  class  after  the  first  meeting  and 
found  it  successful. 

At  another  meeting  Dan  and  I dis- 
cussed adding  to  the  use  of  the  journals 
in  his  class  a tool  for  self  assessment 
Dan  is  very  interested  in  using  life  skills 
instruction  as  a basis  for  class  activity 
and  discussion.  He  also  expressed  his 
belief  that  student  learning  should  be 
goal  oriented,  Le.  that  the  student  should 
consider  his/her  success  in  learning 
English  to  be  linked  to  his  or  her  success 
in  achieving  goals  in  life.  This  is  critical 
since  learning  English  for  the  sake  of 
knowledge  is  not  the  ideal  of  most  ESL 
students  who  need  English  to  survive  in 
a new  country  and  culture. 

In  the  early  Spring  I worked  with  Dan 
to  develop  goals-oriented  techniques 
that  met  the  five  requirements  we  had 
set  at  our  original  meeting.  We  devel- 
oped a monthly  goals  sheet  for  Dan's 
students  to  complete  at  the  beginning  of 
each  month.  The  monthly  time  frame 
was  selected  because  it  was  felt  that  it 
was  long  enough  to  track  achievement, 
yet  frequent  enough  for  the  students  to 
experience  an  ongoing  feeling  of  accom- 
plishments (see  Figure  1). 

For  this  first  month  Dan  selected  five 
life  skills  that  he  was  intending  to  intro- 
duce in  class  during  the  month..  Each 
student  was  handed  a form  with  a 
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personalized  title,  e.g.  "Maria's  Goals — 
May  1994."  Dan  then  asked  the  class  if 
they  wanted  to  learn  how  to  "leave  a 
message."  When  sill  students  understood 
what  was  being  asked  of  them,  they 
answered  the  question  by  tirding  "Y"  or 
"N"  on  their  sheet.  In  this  way  the 
students  were  actually  building  their 
own  curriculum,  taking  charge  of  the 
content  of  their  class.  Those  subjects 
with  low  interest  from  students  might 
not  be  taught  that  month  or  would  be 
taught  through  a teaching  assistant  to 
those  students  that  were  interested. 

» T .1  ii » 1— L.  la...  ...a11 
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they  could  do  the  task  in  English.  They 
did  this  by  drawing  a picture  of  them- 
selves on  a line  scale.  It  indicated  that 
they  had  no  ability  at  all  vis  a vis  this 
task.  If  they  pictured  themselves  on  the 
far  right  it  meant  that  they  could  do  this 
task  with  full  competence  in  English.  Of 
course  any  level  of  competency  in 
between  these  extremes  could  be  repre- 
sented as  well  Again  the  students  were 
providing  the  teacher  with  curriculum 
building  assistance  by  indicating  their 
strengths  and  weaknesses. 

The  students  continued  through  the 
six  pre-selected  tasks  answering  first 
whether  they  wanted  to  learn  the  task 
and  then  how  competent  they  felt  in 
their  ability  to  do  die  task.  The  students 
had  set  and  prioritized  their  goals  and 
the  tasks  to  be  learned,  and  what  would 
be  accomplished  during  the  next  month 
of  classes. 

Finally,  the  students  had  the  opportu- 
nity to  list  up  to  three  other  goals  they 
had.  They  listed  such  things  as  "learn 
how  to  fill  out  job  applications,"  "super- 
market product  containers"  and  "talk  to 
the  teacher  of  my  children."  From  these 


suggestions  Dan  was  able  to  compile  his 
curriculum  for  the  following  month. 
These  suggestions  became  the  tasks  to  be 
included  on  the  next  students'  monthly 
goals  sheet.  Thus,  from  the  point  of 
introduction  of  these  monthly  goals 
sheets,  the  curriculum  could  potentially 
be  100%  driven. 

At  the  end  of  the  first  month  Dan 
recirculated  the  student  monthly  goals 
sheet  that  had  been  completed  previ- 
ously. The  students  had  the  opportunity 
to  reflect  upon  their  perceived  compe- 
tency in  the  task  of  a month  earlier.  They 
were  then  invited  to  draw  a new  picture 
of  themselves  indicating  their  current 
perceived  competence.  All  of  the  stu- 
dents who  had  attended  the  classes  in 
which  the  tasks  were  taught  drew 
pictures  of  themselves  to  the  right  of  the 
original  picture,  i.e.  closer  to  the  goals  of 
complete  competence;  and  there  in  black 
and  white  was  evidence  for  the  teacher 
and  student  alike  that  the  student  was 
moving  closer  to  competence.  Both  Dan 
and  I believe  that  this  periodic  reflection 
is  critical  in  maintaining  student  self 
esteem  and  encouraging  the  student  to 
continue  studying,  to  continue  moving 
closer  to  their  goals. 

As  indicated,  this  monthly  goals 
technique  accomplished  the  five  require- 
ments discussed  at  our  first  meeting.  The 
assessment  was  based  solely  on  student 
reflection  and  reporting.  This  form  of 
self-assessment  could  be  handled  by 
students  with  very  limited  communica- 
tion skills  and  those  with  little  education 
since  there  was  almost  no  writing  to  be 
done  and  the  concept  of  goals  and 
progress  could  be  readily  understood. 
None  found  the  reporting  to  be  childish; 
on  the  contrary,  the  students  enjoyed 
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drawing  the  little  pictures  of  themselves 
and  some  drew  self  portraits.  As  a result 
of  this  technique  the  teacher  was  pro- 
vided with  a document  that  measured 
student  progress  incrementally. 

It  also  went  beyond  the  original  five 
requirements  in  assisting  in  curriculum 
development,  student  involvement  and 
control  of  the  lessons,  and  student  goal 
setting.  Equally  important  is  the  fact  that 
the  technique  takes  very  little  time, 
roughly  one  half  hour  of  class  time  per 
month  and  less  than  one  hour  prep  time. 

I learned  a lot  from  working  with  Dan 
as  part  of  the  Partnership  Project  I 
learned  that  my  experiences  and  difficul- 
ties in  teaching  and  assessing  beginning 
learners  are  not  unique.  I am  now  more 
willing  to  try  new  things  in  my  classes.  I 
was  able  to  share  my  ideas  with  Dan  that 


led  to  the  development  of  a new  tool  and 
I have  learned  from  him  as  well.  I hope 
that  this  relationship  can  continue  and 
further  facilitate  our  challenging  jobs  as 
adult  educators.  In  this  kind  of  relation- 
ship you  are  more  willing  to  take  risks 
and  try  new  things  and  share  ideas.  This 
project  helped  me  become  aware  of  the 
tools  I can  use  with  my  students  and 
how  they  respond  to  them.  Because  of 
this  awareness  my  students  and  I can 
connect  and  build  a better  relationship. 
The  students  can  also  establish  a bond 
with  one  another  and  further  improve 
their  opportunities  to  learn  and  socialize 
in  a school  setting.  I also  realized  the 
importance  of  working  with  students  to 
change  their  attitudes  about  assessment 
and  working  with  them  to  open  up  to 
new  forms  of  assessment 
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Work  in  Progress 

Bottoms  Up:  An  Alternative 
Self-Directed  Readiness  Training  Program 


by 

Don  Robishaw 


Evaluation  and 
Motivational 
Specialist 


In  the  fall  1993  issue  of  Adventures  in 
Assessment,  Vol.  5. 1 discussed  the 
need  for  pregoal  setting  strategies. 

. . . goal  setting  as  a strategy  can  help 
some  learners  take  control  of  their  oum 
learning  and  become  motivated.  But  pre-goal 
setting  strategies  must  be  designed  and 
implemented  to  better  help  adults  develop 
confidence  in  their  own  ability  to  use  goal 
setting  as  a strategy  for  learning. 

I also  offered  one  group  approach  that 
could  possibly  increase  the  chances  of 
goal  setting  being  more  effective  and 
meaningful  in  Adult  Literacy/ Adult 
Basic  Education  (AL/ABE). 

Unfortunately  there  are  some  adult 
learners  who  have  had  little  experience 
in  directing  their  own  learning,  thus  may 
have  problems  with  the  group  investiga- 
tions I suggested,  especially  in  light  of 
the  practitioner's  need  to  give  up  control 
and  ask  the  students  to  direct  or  manage 
their  own  learning. 

Since  writing  that  article  I have  devel- 
oped and  field-tested  a similar,  yet  more 
complex  approach.  The  purpose  of  this 
article  is  to  share  some  of  the  critical 
dialogue  of  the  students  who  partici- 
pated in  a pilot  study  that  took  place  in 
May  and  June  of  1994.  They  participated 
in  a 14  hour  training  program  called 
"Bottoms  Up":  An  Alternative  Self- 
Directed  Readiness  Training  Program. 

If  s goal  is  not  only  to  increase  the 
chances  of  goal  setting  being  more 
effective,  but  also  to  help  prepare  stu- 


dents for  self-directed  learning  strategies 
and  participatory  education  practices 
inherent  in  some  adult  education  pro- 
grams. 

WHY  I DEVI  LOPED  "BOTTOMS  UP" 

I developed  the  "Bottoms  Up"  pro- 
gram to  help  students  move  closer  to 
self-directed  learning  as  one  leg  of  a 
much  larger  framework.  I focused  my 
efforts  on  the  process  that  relates  to  just 
the  learners  and  not  the  practitioners. 

That  full  model  — AGE  (the  Alterna- 
tive Great  Equalizer) — attempts  to 
bring  students  and  practitioners  together 
as  equals.  For  true  participatory  educa- 
tion (not  manipulation)  to  take  place,  the 
following  caveats  must,  at  some  point, 
be  addressed: 

1.  Both  the  students  and  the  practitio- 
ners have  to  have  the  same  leverage 
and  power  for  true  participatory 
education  to  be  successful 
(Demardno,  A.M.,  1994) 

2.  Practitioners  need  to  play  the  role  of 
an  observer  and  identify  students  who 
need  to  participate  in  an  intervention 
in  order  to  shed  their  excess  baggage. 

3.  Most  adult  educators  need  to  partici- 
pate in  their  own  brand  of  Readiness 
Training  Program  or  in  other  words, 
an  unlearning  process. 

4.  Both  students  and  practitioners  need 
to  first  take  part  in  a separate  reflec- 
tive processes  in  order  to  later  come 
together  as  "near"  equals. 
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5.  This  "ideal  state"  takes  a considerable 
amount  of  time,  in  feet  it  has  to  be  a 
continuous  lifelong  - unending  pro- 
cess. 

In  this  article  I am  presenting  several 
student  produced  critical  dialogues  from 
the  initial  field-test  of  the  first  "leg"  of 
the  AGE  model  - "Bottoms  Up."  In  two 
previous  articles  in  the  All  Right  News, 
January,  1994,  'Heading  and  Writing 
Professionals  or  Literacy  Workers"  and 
May,  1994,  "Diversity  or  Conformity,"  I 
suggest  possible  topics  for  practitioners 
who  participate  in  an  alternative  un- 
learning  procsss,  ths  second  leg.  Perse  n~ 
ally  I would  love  to  see  some  of  my 
colleagues  who  responded  in  the  last 
issue.  Responding  to  the  Dream  Conference, 
to  work  on  developing  a curriculum  for 
the  second  leg  of  the  AGE  model 

THI  ALTERNATIVE  ASSESSMENT 
CONNECTION 

Another  incentive  for  developing 
"Bottoms  Up"  comes  by  way  of  all  the 
progressive  programs  and  practitioners 
who  are  developing  exciting  and  innova- 
tive self-directed  learning  tools  and 
methods  used  to  assist  students  in 
learning  how  to  manage  their  own 
learning.  Why,  we  even  have  a journal 
devoted  to  a single  aspect  of  external 
self-directed  learning  - Adventures  in 
Assessment.  There  is  a lot  of  potential  for 
these  innovations  that  help  students: 

1.  Diagnose  their  own  learning  needs, 

2.  Develop  and  set  goals, 

3.  Identify  resources  for  learning, 

4.  Choose  learning  strategies, 

5.  Assess  their  own  learning,  and 

6.  Assist  practitioners  and  learners  in 
designing  and  evaluating 


adult  participatory  education  pro- 
grams. 

"Bottoms  Up"  first  deals  with  the 
internal  side  of  self-direction 
(Robishaw,  D.L.,  1993),  prior  to  adults 
learning  to  manage  their  own  learn- 
ing. It  later  guides  and  connects  the 
learners  to  these  new  participatory 
education  practices  or  alternative 
tools. 

"Bottoms  Up"  may  just  be  that  "miss- 
ing link"  that  brings  all  the  other  mosaic 
fragments  into  focus.  It  helps  learners 
develop  a more  positive  self-concept 
through 

• unlearning  the  shame  issue  of  return- 
ing to  school  as  an  adult; 

• unlearning  the 'blaming  themselves' 
mentality; 

• recognizing  and  giving  themselves 
enough  credit  for  overcoming  barriers; 

• developing  the  self-confidence  needed 
to  feel  capable  of  doing  academic 
work  on  their  own  or  in  a group,  thus 
being  able  to  manage  their  own 
learning  and  connecting;  and 

• making  clearer  the  purposes  of  these 
alternative  tools,  strategies  and  meth- 
ods, as  well  as  preparing  them  for 
participatory  education  programs. 

Specifically,  some  students  need  help 
developing  self-motivation  for  learning 
and  then  channeling  that  motivation  into 
the  persistence  needed  to  becoming  self- 
directed.  After  students  begin  the  critical 
reflection  process  inherent  in  "Bottoms 
Up",  they  will  gradually  gain  the  self- 
motivation  and  confidence  needed  to 
become  the  more  complete  form  of  self- 
directed  learner,  which  includes  lifelong 
learning. 


Both  the 
students  and  the 
practitioners  have 
to  have  the  same 
leverage  and 
power  for  true 
participatory 
education  to  be 
successful. 
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Students  participating  in  "Bottoms 
Up"  will  have  the  space  to  deconstruct 
their  internal  oppression  and  past 
experiences  with  schooling,  to  critically 
reflect  upon  the  educational  system  and 
other  issues  and  to  participate  in  a 
dialogue  with  their  cohorts.  This  self- 
directed  readiness  program  offers  the 
students  opportunities  to  pursue  the 
knowledge  needed  to  overcome  the 
above  mentioned  dispositional  (internal) 
barriers  that  may  interfere  with  meeting 
his  or  her  needs,  goals  and  interests  in 
future  programs  or  educational  projects. 
After  participating  in  the  program 
students  may  then  be  able  to  make  the 
appropriate  attitudinal  and  behavioral 
changes  necessary  in  order  to  accept 
learning  in  programs  as  a viable  option. 

TNI  DSVKLOPMKNT  PROCESS 

The  development  of  the  program  itself 
has  been  a five-year  process.  I had 
previously  conducted  inquiries  into 
resistance,  persistence  and  the  situ- 
ational, institutional  and  dispositional 
barriers  adults  faced  when  returning  to 
school.  I eventually  narrowed  that  focus 
to  specifically  look  at  dispositional 
barriers.  I combined  what  I learned 
through  a brief  review  of  the  literature, 
directly  through  research  projects  con- 
ducted with  students,  and  from  my  own 
self-reflections  and  the  creation  of  a 
staged  developmental  model  based  on 
my  own  growth  experiences,  including 
taking  part  in  all  the  activities  that  came 
with  the  turf,  including  dropping  out  of 
school  (twice).  An  elaborate  evaluation 
plan  was  also  designed  and  conducted 
throughout  the  entire  development 
process,  but  is  not  included  in  this 
article. 


INK  PROGRAM 

There  has  been  one  trial  run  of  "Bot- 
toms Up"  thus  feu,  with  two  partici- 
pants, one  male  and  one  female.  The 
participants  spent  about  14  hours  in 
seven  workshops.  In  each  workshop  we 
examined  two  different,  yet  related 
issues  in  AL/ABE.  In  each  of  the  seven 
components,  the  students  were  first 
asked  to  share  with  each  other  and  the 
facilitator  their  opinions,  perspectives 
and  personal  experiences  on  these 
issues.  I served  as  the  facilitator  and 
begem  the  process  by  presenting  the 
topics  for  discussion  with  a brief  state- 
ment, followed  up  by  several  discussion 
and  probing  questions.  Later  different 
opinions  and  perspectives  were  added  to 
the  discussion  in  the  form  of 

• Fictitious  case  studies  and  role  plays 

(dyads) 

• Brainstorms 

• Games 

• Other  activities 

To  make  each  separate  workshop  a 
success  and  more  valuable  the  partici- 
pants were  encouraged  to  talk  about 
their  personal  experiences,  perspectives, 
opinions  and  values,  as  they  related  to 
the  topics.  Participants  were  also  encour- 
aged to  go  beyond  their  initial  position 
by  comparing,  contrasting  and  weighing 
other  positions  or  views  carefully 
against  their  own. 

DYADS/CRITICAL  DIALOGUES 

The  first  goal  was  to  get  the  partici- 
pants' perspectives,  then  to  introduce 
outside  perspectives  through  the  dyads 
( see  Figures  1 and  2)  and  then  to  continue 
the  discussion  process  from  there.  The 
dyad  and  the  following  critical  dialogue. 
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Resistance  and  persistence  in  schools,  make 
up  one  component. 

All  the  critical  dialogues  in  this  article 
were  extracted  from  the  14  hours  of 
taped  discussions  that  took  place  in  May 
and  June  of  1994  at  an  adult  education 
program  located  in  Massachusetts.  The 
names  of  the  participants  and  their 
respective  program  have  been  changed 
to  protect  their  anonymity. 

Karla  is  about  30  years  old  with  three 
children.  She  had  been  enrolled  in  a 
couple  of  other  adult  schools  prior  to 
enrolling  in  her  current  program.  She 
did  not  complete  formal  schooling. 

Sam  is  about  25  years  old  and  single. 
He  had  been  enrolled  in  this  program  for 
six  months  prior  to  joining  "Bottoms 
Up".  Sam  did  complete  high  school. 

The  boxed  examples  on  the  next  two 
pages  are  two  fictitious  case  studies 
(dyad)  used  in  the  third  component  - 
Resistance  and  Persistence  in  Schools. 

cam  CAL  DIALOGUES 

COMPONENT  3 - RESISTANCE  AND 
PERSISTENCE  IN  SCHOOLS 

[Do  some  students  resist  schooling?  How?] 

Karla:  Yes,  they  do.  By  not  showing  up. 
Sometimes  when  you  do  show  up  you 
get  a little  panicky,  you  get  a little 
scared.  There  are  some  kids  who  tend  to 
follow  their  parents.  Like  if  their  parents 
worked  on  a farm  and  that  is  what  the 
father  is  looking  for  in  the  boys...  and 
the  son  doesn't  want  to,  he  wants  to 
become  a lawyer  or  a doctor  or  just 
wants  to  be  nobody.  Sometimes  when 
you  try  to  make  your  son  take  over  your 
business  or  be  something  you  was  they 
end  up  on  the  streets  and  homeless 


sometimes.  I've  seen  it  happen. 

Sometimes  they  [teachers]  force  it, 
sometimes  you  have  a teacher  who 
won't  say  it  out  loud  but  that  [soft  voice] 
John  belongs  over  here  or  Jane  belongs 
over  here.  By  color.  John  should  be  the 
machine  operator  or  Joe  should  be  in 
wood  [shop]  because  some  black  people 
should  be  contractors  - because  that  is  all 
they  deserve.  He  doesn't  belong  here 
because  they  have  structured  classes  for 
lawyers,  doctors,  teachers . . . Sometimes 
the  teacher  offers  scholarships  to  the 
kids.  Sometimes  they  think  that  this 
white  student,  Jane  deserves  it.  [soft 
voice]  "Jane,  take  this  home  to  mother 
and  father  and  don't  tell  nobody."  They 
do  that.  They  do,  it  happens. 

[Any  other  examples ?] 

Sam:  Someone  who  goes  in  and  is  a 
clown  or  something  probably.  Kids  do  a 
lot  of  things  to  get  the  class  laughing  or 
something. 

Not  coming  in,  pretending  to  be  sick, 
by  getting  into  fights,  by  being  bullies, 
taking  money  for  lunch.  Skipping  class 
and  being  in  the  hallways  walking 
around. 

They  have  cliques.  My  sister  use  to  go 
to  school  but  she  had  to  be  there  early  to 
see  her  clique.  Some  people  don't  go  to 
class  they  just  hang  out 

Karla:  On  my  way  to  school  today  I saw 
a young  lady  coming  from  one  of  the 
schools.  She  had  on  a dress  tighter  than 
my  finger.  You  could  see  her  figure  in  it. 
High  heels  - like  they  had  been  "pulling 
tricks".  They  are  presenting  themselves 
as  if  you  can  approach  me,  here  I am.  No 
female  should  be  allowed  to  come  to 
school  now  a days  - 1994  - dressed  that 
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I.  JOHN  DOE  ON  GETTING  ANGRY  OR  "IT  WASN'T  LEARNING  THAT  I DIDN'T 
LIKE;  IT  WAS  SCHOOLING" 


Vocabulary:  1.  Rebel  2.  Capable  3 . Authority  4 . Identify  5 . Role 

Question:  John , worm  you  a resistor  or  a rebel  in  school t 

Was  I really  such  a troublemaker  as  a kid?  Did  I really  have  behavioral  problems  like 
they  used  to  tell  me?  Why,  for  a while  there  they  even  thought  I had  a kind  of  learning 
problem. 

Sure  I acted  up  a little,  but  I still  liked  to  learn  new  stuff.  Even  outside  of  school  I liked 
to  learn  new  things.  You  know,  play  with  the  bugs  and  birds  and  other  creatures.  I liked  to 
learn  - still  do  in  fact.  I was  even  capable  of  the  academic  stuff  too.  Why  didn't  I stay? 
Was  it  my  fault?  There  I go  blaming  mvself  again.  It  wasn't  my  fault.  They  did  a pretty 
good  job  of  "locking  me  out*. 

Ldidn't  like  school  and  told  the  teachers  I didn't  like  school.  Why  didn't  they  change 
the  place  if  75%  of  us  didn't  like  it  and  left?  Sure  I resisted  the  school  system  and  author- 
ity. They  were  the  authority  and  were  not  going  to  listen  to  me  or  the  other  kids. 

We  got  no  respect.  Oh  they  respected  the  kids  who  came  in  by  bus,  but  not  us.  Why 
should  I try  so  hard  to  be  like  them  if  they  didn't  respect  me.  The  guys  in  the  "projects* 
respected  me  cause  I could  fight  pretty  good.  They  respected  me  because  I told  the 
teachers  that  school  *&##@.  So  sure  I resisted  schooling.  Nobody  in  my  neighborhood 
was  going  anywhere  even  if  they  finished  school.  It  didn't  make  any  difference  to  anybody 
if  you  stayed,  and  it  didn't  make  any  difference  if  you  left.  Maybe  if  it  did  make  a differ- 
ence to  someone,  but  it  didn't.  So  now  I've  got  this  "monkey  on  mv  back*  because  I left 
early,  but  what  can  I do  about  it  now? 

I can  tell  you  what  was  wrong  with  school  then  and  I can  tell  you  what  is  wrong  with 
school  now  - no  respect.  We  got  no  respectl  Kids  need  to  be  respected  once  in  a while 
too.  If  they  don't  get  respect  in  school  they  will  find  someplace  else  to  get  respect.  If  I was 
a resister,  I wasn't  the  only  one. 

Questions: 

1 . Was  John  a resister?  What  in  the  story  says  that  he  was  a resister? 

Can  anyone  identify  with  John? 

2.  What  caused  him  to  resist? 

3.  John  talks  about  a "monkey  on  his  back".  What  is  it?  What  can  he  do  about  it? 

4.  Are  there  other  roles  that  the  school  system  tries  to  force  people  to  accept  because  of  who 
they  are?  Examples  Sex,  class,  race  .... 

5.  What  are  some  of  the  good  points  that  a resister  has?  Can  these  good  points  be  directed 
into  something  positive  as  an  adult  learner?  What  are  some  of  the  bad  points  about  being  a 
resister? 
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HOURS  R 

II.  TONYA  SMITH  ON  DEVELOPING  PERSISTENCE:  SURE  THERE  ARE  ALWAYS 
SETBACKS  BEFORE  SUCCESS 

Vocabulary : 1 . Possessed  2.  Actually  3 . Succeed  4 . Environment  5*  Support  6 . Setback  7 \ 
Stubborn  8.  Effort  9.  Invented 

Question:  Tonya , were  you  a persister  in  school? 

Was  I really  a persister  in  school?  Yes,  I had  to  be.  My  parents,  teachers,  coaches  and 
classmates  would  nat  let  me  be  anything  else.  Far  awhile  I was  so  possessed  with  doing  well  in 
school  it  actually  made  me  become  sick.  I would  da  anything  to  succeed. 

My  friends  in  school  were  so  much  like  me  too.  The  teachers,  my  parents  and  my  friends 
always  expecting  me  to  do  well.  They  were  all  just  like  me.  But  I liked  it. 

I 1;L~J  T U.  Anwir/MMnAnf  u/nc  nmrrf  Hura  wn«  nlwm/*  tn  mnrh  sunnort  for  mv  soddal 

I IMV  VM.MWM.H.M.  o - — - r I - / » 

projects.  Learning  at  a young  age  just  "grabbed  me"  by  the  pants  and  wouldn't  let  go. 

Oh  I wasn't  perfect.  There  were  always  setbacks  before  success  came.  But  I was  very 
stubborn.  I wouldn't  be  defeated.  I learned  from  my  father  early  an  about  Tom  Edison.  He 
invented  the  first  light  bulb.  He  invented  many  other  things  too.  He  made  more  mistakes  in  life 
than  anybody.  But  he  persisted.  Why  he  once  made  9,999  mistakes  in  a row.  He  kept  "getting 
up  aff  the  ice".  Finally,  an  the  10,000th  try  his  little  talking  machine  said  the  words,  "Mary  had 
a little  lamb."  If  there  is  ane  word  that  means  more  ta  me  than  anything  it  is  persistence. 

Yau  have  to  know,  that  when  I went  to  school  most  women  were  being  raised  ta  stay  home 
and  become  good  housewives.  Well  that  wasn't  for  me.  Much  later  I did  all  that  stuff  but  that 
was  nat  an  my  mind  as  a kid  in  school.  I knew,  if  I persisted  and  worked  hard  that  I would  be 
rewarded  for  my  efforts.  It  is  OK  to  stumble  and  "fall  down"  and  make  mistakes,  but  you  have 
to  keep  "getting  up  off  the  ice".  There  is  a big  world  out  there  and  you  have  ta  "unlock  the 
door".  Nobody  is  going  ta  help  you  "apen  the  door".  I didn't  have  any  extra  tutoring  like  some 
of  my  classmates.  Oh  yes,  I did  it  all  on  my  own  and  I did  it  my  wayl 

Schools  offer  equal  opportunities  for  all.  I believe  that  the  average  person  can  fulfill  most 
goals.  Anybody  can  do  what  I did,  as  long  as  they  develop  persistence  and  work  hard.  I knew 
that.  My  parents  knew  that  too.  Everybody  knew  that.  Some  people  persisted  and  others  wasted 
their  time.  Yes  I was  a persister,  now  I am  a full  professor. 

Questions: 

1 . Was  Tonya  a persister?  What  in  the  story  points  out  that  she  was  a persister? 

2.  What  caused  her  to  persist? 

3.  Why  did  Tonya  like  school? 

4.  What  did  Tanya  mean  when  she  said,  "learning  grabbed  me". 

5.  What  are  some  of  the  good  points  that  a persister  has?  Can  these  be  directed  into  something 
positive  for  adult  learners?  What  are  some  of  the  bads  points  af  being  a persister? 
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way.  If  you  allow  them  to  do  that,  they 
are  going  to  get  away  with  anything  and 
they  are  going  to  feel  like  they  can. 

When  they  start  they  are  going  to  start  to 
rebel,  like  they  are  doing  "anyways". 
[What  cause  some  students  to  resist  school- 
ing?] 

Karla:  I've  been  there.  There  were  two 
monkeys  on  my  back.  I'll  say  three.  One 
in  school  - teachers,  coming  home  and 
classmates.  There  was  no  teacher  sup- 
port, student  support  and  family  sup- 
port. You  have  got  to  find  that  support 
somewhere. 

Sam:  Making  fun  of  you.  My  father  used 
to  make  fun  of  me  when  I was  a kid. 
"Rank"  on  me  and  stuff.  My  brothers 
would  "rank"  on  me. 

[What  about  the  ones  who  persist ?] 

Karla:  When  you  find  out  that  your  child 
likes  school  and  they  are  sticking  to  it 
and  learning  is  interesting  to  them  and 
they  keep  going  you  have  no  problems. 

It  is  not  hard  to  steer  your  child  in  the 
right  direction  either.  They  are  mainly 
steering  themselves.  You  have  already 
put  the  work  into  it.  You  have  done  your 
job. 

[What  are  some  examples ?] 

Sam:  Steve . He  wore  glasses 

and  the  whole...  a "nerd".  An  (A)  stu- 
dent, triple  (A)  student,  whatever  you 
want  to  call  it.  Everything  comes  easy  for 
him.  He  just  wants  to  study.  He  doesn't 
have  a girl,  doesn't  want  a girl.  But  he  is 
really  smart  Comes  in  early,  wants  more 
homework,  likes  everybody,  a nice  guy. 


with  a neat  little  tie.  Participates  more 
like  in  chess  find  different  kinds  of  clubs. 

[What  causes  them  to  persist ?] 

Sam:  If  they  like  school  they  have  fun.  If 
you  don't  like  it  or  have  a bad  time  you 
are  not  going  to  have  a very  good  time. 
They  get  picked  on.  [But  when  they 
become  adults...]  you  want  to  be  them; 
they  get  the  money. 

Karla:  Parents.  They  were  persistent  in 
making  sure  that  she. . . helping  her  to 
reach  her  goals.  The  student  also,  her 
classmates,  her  teachers.  It  has  got  to  be 
everybody. 

Sam:  A happy  childhood.  Her  mother 
and  father  treated  her. . . . She  liked 
learning,  she  liked  school 

[What  are  some  characteristics  of  a 
persister?] 

Karla:  On  time  for  class. 

Sam:  Don't  let  things  bug  'em. 

Karla:  Get  there  homework  done,  ask  for 
more  work,  address  problems  in  schools, 
be  more  open  and  persistent. 

[Are  there  any  bad  characteristics  of  a 
persister?] 

Karla:  Yes  there  are!  Sometimes  a stu- 
dent who  is  a persister  or  perfect  tends 
to  get  on  your  damn  nerves.  Sometimes 
they  start  trouble.  Yes  they  do.  Some- 
times their  head  gets  real  big  - chip  on 
the  shoulder  - and  your  head  is  up  in  the 
air. 

Sam:  Arrogant! 

Karla:  Better  than  anybody  and  they 
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realize  it.  Some  kids  they  are  very  mean 
and  nasty. 

Sam:  Some  kids  take  care  of  them  real 
quick,  trip  them  or  put  gum  on  their 
seats  or  put  something  [signs]  on  their 
butts  - kick  me  or  something  like  that. 

No  one  needs  it  [their  arrogance] 

[ Can  this  characteristic  relate  to  adult 
students?] 

Karla:  Adult  students  who  are  going 
back  to  learn  again  can  benefit  from 
when  they  were  a child  [experiences]. 
That  support  for  me  as  a child  wasn't 
there. 

Sometimes  when  I come  to  school 
now  as  an  adult  and  think  back,  that  if  I 
had  thought  about  it  more  clearly  and 
stuck  to  it  more  when  I was  younger.... 

If  there  is  something  wrong  in  the  home 
then  the  child  will  not  be  able  to  function 
right  in  school  It  comes  from  home.  It  is 
never  to  late  to  learn,  but  it  is  best  to  get 
it  while  you  are  young.  It  is  never  too 
late. 

Like  now  I go  to  the  adult  learning 
class  and  everybody  is  equal  - cause 
everybody  understands  everybody. 

[Solidarity?] 

Sam:  Yeah,  kind  of  like  that 

Karla:  It  sends  an  electricity  through  my 
system  and  I feel  good.  When  I don't 
attend  class  I hurt  myself  more  It  is  not 
being  done  purposely.  We  all  have  to 
study  more  to  learn. 

Sam:  You  know  the  teachers  in  there  are 
pretty  nice  too.  They  are  real  different, 
they  are  easy  going.  They  are  not,  you 
know, ...  sometimes  teachers  have  that 


little  [chip  on  their  shoulder].  It  seems 
like,  yes  a chip  on  the  shoulder  - arro- 
gance. Tm  better  than  you  because  I'm 
teaching  you."  These  people  [in  this 
adult  program]  are  like  one  of  you. 

ADDITIONAL  DIALOGUES 

For  the  sake  of  space,  only  the  dia- 
logues between  the  participants  follows 
from  other  discussions.  Again  these 
critical  dialogues  are  in  the  participants' 
own  words.  Both  participants  were 
involved  in  the  editing  process.  They 
were  offered  the  opportunity  to  "edit 
out"  any  comments  that  they  were 
uncomfortable  with. 


COMPONENT  4- 
RESISTANCE  & PERSISTENCE 
IN  ADULT  EDUCATION 


[Why  do  some  adults  come  back  and  others 
don't?] 


Karla:  Some  adults  need  to.  It  took  me  a 
while  to  find  a program.  This  program  is 

right  for  me.  The program 

wasn't.  It  was  self  work  - you  worked  on 
your  own.  It  just  wasn't  there  for  me. 

I've  been  there  twice.  Then  I went  to 
another  program  that  wasn't  for  me. 

This  program is  excellent.  I like  it. 

From  this  one  I achieved  a little  bit  more. 

Some  of  them  feel  it  is  too  late  or  they 
are  too  old.  Some  of  them  just  don't  care. 
"Why  should  I learn  how  to  read.  What 
am  I going  to  benefit  from  it 

Sam:  Because  they  have  kids  - barriers  - 
a wife,  a job.  [are  kids  a barrier?]  No,  but 
in  this  day  and  age  you  might  need  two 
jobs.  Something  has  got  to  be  done  with 
those  kids.  You  do  not  want  a stranger 
down  there.  Do  you  know  what  I am 


Like  now  I go 
to  the  adult 
learning  class  and 
everybody  is  equal 
- cause  everybody 
understands 
everybody. 
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I had  kids  and 
they  told  me 
that  I zoos  more 
important  than 
my  children, 
and  that  is  a lie. 
Your  education  is 
important  and  not 
your  children? 

She  told  me  that. 
That  is  what  she 
told  me. 


saying?  I don't  think  it  is  a barrier,  you 
have  to  do  what  you  have  to  do.  School- 
ing is  out  there  some  place.  If  you  can  do 
it,  you  can  do  it.  [some  things  are  more 
important  than  school]  Yeah,  putting 
clothes  on  the  kids.  They  are  your  kids, 
you  had  them.  If  you  didn't  want  them 
you  wouldn't  have  them.  I remember 
when  I was  growing  up  we  had  a wise 
mouth  and  would  say,  what  do  you 
think  you  own  us?  When  you  get  older 
you  start  to  think,  maybe  they  could 
own  you  - just  a tiny  little  bit. 

Karla:  Some  people  can't  make  it  be- 
cause they  simply  don't  know  how  to 
take  the  bus.  There  are  some  that  can't 
read  at  all  so  it  looks  different,  especially 
if  you  come  from  a different  country.  But 

I came  from [USA].  It  is  just 

that  the  words  look  different  to  me.  I 
couldn't  articulate  it  [the  problem],  but 
now  I can. 

[What  about  inside  barriers  - psychological 
and  emotional  stuff?] 

Sam:  Attitude,  probably. 

Karla:  Self  conscious  and  self  confidence. 
They  have  to  have  enough  confidence  in 
themselves.  Some  have  a handicap  and 
they*  re  afraid  of  that  too.  They  are 
embarrassed. 

Sam:  [Some]  They  are  bullheaded. 

Karla:  Some  just  lay  in  bed  all  day  and 
feel  comfortable  doing  that  - no  job,  no 
life,  no  nothing.  Why  interfere  and  go  to 
school  and  get  an  education. 


[Developing  persistence  in  Adult  Education 
programs ] 


Sam:  Keep  on  coming! 
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Karla:  Try  not  to  miss  and  if  you  do, 
study  at  home  - study  more  at  home. 
When  you  study  more  at  home  it  helps. 
It  goes  a long  way. 

[Critical  vocabulary  words ] 

Karla:  Criticize  - Some  people  come  here 
and  they  are  not  employed.  They  are  on 
AFDC  or  Social  Security  and  others 
work.  They  [some  of  those  who  work] 
seem  to  think  that  they  are  more  better 
than  you.  Anyone  can  feel  the  tension  It 
exists.  It  happens. 

Sam:  Some  of  them  are  just  mean. 

[What  about  other  programs ?] 

Karla:  I dropped  out  of  the 

program  and  went  back  to  the . 

program.  I had  kids  and  they  told  me 
that  I was  more  important  than  my 
children,  and  that  is  a lie.  Your  educa- 
tion is  important  and  not  your  children? 
She  told  me  that  That  is  what  she  told 
me. 

Sam:  You  are  important  too,  though. 

Karla:  I am,  but  I had  those  kids.  They 
didn't  ask  to  come  here.  If  I have  to  drop 
out,  which  I do  not  want  to....  if  I feel 
that  I have  to  drop  out  I will  drop  out. 
They  are  probably  saying,  she  wants  to 
drop  out  and  lay  up. 

[If  the  problem  is  the  program,  how  do  they 
need  to  change?] 

Karla:  Some  of  the  programs  really  need 
to  stick  to  the  level.  "Well  we  don't  have 
this  here  for  you,  but  we  have  got  it  for 
Maria  who  comes  from  Puerto  Rico."  I 
was  in  a program,  it  really  made  me  mad 
when  I realized  that  she  was  in  a level 
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that  I should  be  at.  They  had  me  doing 
math  and  she  was  reading.  I said  why 
are  you  reading?  I'm  reading  so  I can 
learn  how  to  speak  better  English.  It  was 
the  same  level  I should  have  been  at.  The 
only  thing  is  that  she  is  in  it  for  a differ- 
ent reason.  I'm  in  it  because  I need  to 
learn  [to  read].  There  are  some  people 
who  don't  know  how  to  be  a teacher.  If 
you  are  not  getting  it,  it  makes  a lot  of 
difference.  It  is  not  fair  to  you.  Who 
wants  to  come  into  class  everyday  - 
adding?  Do  you  know  what  I'm  saying? 

I want  to  read.  The  most  important  thing 
to  Hie  is  leading.  TTiey  Siiould  try  to 
meet  everybody's  goals  [needs]. 

Sam:  Boy,  you  are  really  a very  angry 
women. 

Karla: program  should  be 

mainly  for  people  who  want  their  GED. 
They  are  a little  bit  unorganized.  That  is 
a waste  of  funding.  It  is  not  set  up  right 

and program  is.  It  offers  a lot  for 

everybody's  level 

Sam:  They  [ program  teachers] 

are  laid  back  too.  Their  nose  "ain't"  up 
in  the  air.  Teachers  should  be  less 
uppity. 

Karla:  The  program  I was  in,  they  were 
truly  uppity. 

[How  else  should  programs  change?] 

Sam:  Programs  got  to  do  their  own 
things. 

Karla:  I don't  think  they  should  promote 
someone  that  they  know  is  not  ready  for 
it.  That  is  wrong  and  unfair. 

Sam:  It  is  a time  bracket  too.  When  the 
government  is  funding  it  there  is  always 
a time  bracket.  Like  48  weeks  or  24 
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weeks  or  30  weeks  or  something  like 
that. 

Karla:  Even  though  the  government  is 
funding  it,  it  is  still  unfair  to  us. 

There  is  a time  bracket  and  that  is 
true,  but  they  should  be  open  enough  to 
let  us  know  that  this  is  not  for  you.  That 
there  are  other  programs.  If  they  are  that 
concerned,  but  some  of  them  are  not. 

I found  out  about  this  program 

through  the program  - through  a 

student.  Then  my  husband  came  in  and 
found  out  about  it.  He  brought  me  here 
and  got  me  involved.  He  was  looking  for 
programs  for  me.  He  asked  around.  We 
were  both  looking. 

The program  is  strictly  self- 

dependent,  you  have  to  do  it  on  your 
own.  The  teachers  can  not  help  you  as 
much  either.  They  make  you  take  a test.  I 
took  a test  I don't  even  know  what  the 
hell  I marked  off.  I could  have  been 
marking  off  something  and  [then]  they 
come  and  pick  me  up  and  wrap  me  up 
in  padding  and  beat  me  up.  That  is 
unfair.  Sometimes  the  government 
"sticks  it"  to  you. 

It  is  there  to  help  you.  It  is  just  that  the 
helpers  that  they  have  need  help  them- 
selves. They  do.  When  I was  going,  this 
women  was  a college  something  and  she 
was  all  into  herself. 

Sam:  Well  that  is  college.  They  are 
"profethers"  [poking  fun  at  elitism].  As 
soon  as  you  get  to  be  a "profether"  ha, 
ha,  ha. 

Karla:  So  I left.  They  were  really  uppity. 

Sam  and  Karla  - Barriers:  Transporta- 
tion, knowing  how  to  get  there,  finding 
out  if  there  are  programs  for  you  and 
your  children,  finding  the  right  pro- 
gram, self  doubt,  respecting  authority  of 
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teachers,  sometimes  depression,  fear, 
peer  pressure  and  courage. 

[What  can  students  do  themselves  in  order  to 
develop  persistence?] 


[Make  sacrifices] 

Karla:  You  got  to  take  that  time  to  sit 
down  and  study. 


Feel  proud  about 
what  you  have 
achieved.  Feel 
proud  about 
pushing  yourself. 
When  you  push 
yourself  and  once 
you  get  it  you  can 
feel  proud  about 
it. 


Sam:  Study! 

Karla:  Stick  to  it  • study.  Be  persistent 
about  asking  about  programs.  Be  persis- 
tent about  getting  involved  in  programs. 
Be  persistent  about  getting  there.  Be 
persistent  about  finding  out  what  pro- 
grams your  kids  can  get  involved  in 
while  you  are  at  the  program.  Be  persis- 
tent in  knowing  everything.  Ask  die 
teachers  questions.  Ask  the  students. 

Sam:  Get  tokens,  get  a license  and  try  to 
get  a descent  car.  If  you  don't  have  die 
money  try  to  borrow  somebody's  car  or 
go  for  a walk. 

Karla:  Find  out  the  location  and  make 
sure  it  is  not  rough.  Be  persistent  about 
getting  there.  Be  on  your  toes. 

[Let  teachers  know  how  they  learn  best  and 
how  they  feel  about  learning.] 

Karla:  Yes  it  is  important  because  if  you 
don't  say  anything  you  will  be  in  the 
same  level  you  are  in.  It  is  important  so 
you  can  reach  the  level  you  want  to 
reach. 

Sam:  Well  if  the  teachers  are  ok  with  it. 
Some  teachers  don't  like  that. 


[Make  school  a priority] 

Karla:  Especially  if  you  are  not  working. 
Those  who  do  not  have  a job,  that  can  be 
a priority.  It  am  be  second  priority  if  you 
have  children. 


[Wait  for  the  learning  to  grab  you?] 

Karla:  No,  never  wait  for  anything  to 
grab  you,  you  grab  it!  Cause  if  you  wait 
for  someone  to  come  and  give  you 
something...  That  is  like  waiting  here 
and  Sam  will  bring  a teacher  by  for  me. 
Never  wait  and  be  persistent 

[Feeling  proud?] 

Karla:  Feel  proud  about  what  you  have 
achieved.  Feel  proud  about  pushing 
yourself.  When  you  push  yourself  and 
once  you  get  it  you  can  feel  proud  about 
it.  [Is  that  the  same  as  chip  on  the  shoul- 
der?] 

Sam:  No  I don't  think  so.  Chip  on  the 
shoulder,  you  think  you  are  something. 
Proud  is  you  feel  good.  Chip  is  outside. 

[Open  up] 

Sam:  I don't  know  about  that  Open  up  a 
bit  Maybe  a litde  bit,  but  not  a lot 

Karla:  To  me  it  means  to  be  more  open 
about  yourself.  Why  you  are  in  the 
program.  Open  up  more  about  your  lack 
of  reading. 

Sam:  That  is  alright,  but  I wouldn't ...  I 
don't  trust  people.  I know.  I got 
"screwed"  too  many  times. 

Karla:  I have  a problem  with  trusting 
people  - even  trusting  my  spouse. 

Sam:  People  in  particular.  Some  people, 
they  are  all  hypocrites. 
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Karla:  I'm  not  really  ready  to  let  my 
guard  down.  I'm  not  ready  to  let  some- 
one come  in  and  help  me  too  much.  I am 
independent  Sometimes  it  is  alright  to 
let  someone  help  and  not  be  so  indepen- 
dent. But  when  I feel  that,  that  person  is 
helping  too  much  and  taking  away  my 
independence  I get  scared,  panicky, 
defensive. 

[Keep  a dream] 

Sam:  Well  whatever  you  want  to  be. 
Keep  the  spirit. 

COMPONENT  5 - PERSISTENCE  & 
MANAGING  YOUR  OWN  WORK 

[Besides  yourself,  who  else  can  help  you 
develop  persistence?  How?] 

Karla:  Teachers.  By  asking  you  if  you 
took  out  a book  to  read  or  did  you  study 
at  home.  Find  out!  And  if  it  is  not  being 
done,  be  more  persistent  about  it 

Sam:  Your  wife,  spouse,  mother  and 
father. 

Karla:  My  son  helps  me.  He  says  ma,  are 
you  going  to  read  a book  today?  Some- 
times I say  no.  He  says  why  not?  What 
are  you  doing?  I'm  usually  watching  TV 
when  I should  be  reading  a book  or 
practicing  on  the  computer.  I forget 
words  a lot  I forget  what  I'm  writing 
and  I forget  how  to  spell.  It  is  usually  the 
same  words.  I ask  my  son  to  spell  for 
me.  He  says  you  are  still  asking  me  the 
same  words.  He  says  to  think  about  it. 

So  I have  to  really  get  into  what  I'm 
doing.  I need  to  do  that. 

Sam:  Give  you  a good  kick  in  the  butt 
and  say  get  going.  [Is  that  helpful?]  Oh  I 


think  it  is.  The  kick  would  hurt  a little 
bit.  Why  don't  you  just  unplug  the  TV 
and  read? 

Karla:  I don't  have  to  unplug  it.  I just 
assign  myself  a seat  at  the  table. 

[What  about  self-directed  learning?] 

Sam:  You  have  to  make  it  to  the  class. 
You  have  to  be  in  the  class  before  you  do 
anything  "anyways".  You  have  to  get  in 
the  class  and  learn. 

Karla:  It  means  doing  your  own  work. 
You  are  going  to  a class  and  managing 
your  own  work.  You  are  doing  that 
assignment  that  they  are  giving  you.  It 
can  be  math,  it  can  be  science,  it  can  be 
about  words  or  compound  words, 
writing  a letter.  It  could  be  adding  & 
subtracting.  But  it  is  a program  you 
enter  into  and  you  are  doing  it  on  your 
own. 

[Is  there  a connection  between  persistence 
and  Self-directed  learning?] 

Sam:  Yeah,  persistence  going  to  class. 
You  want  to  learn.  Keep  on  coming  to 
class.  Make  the  class  interesting.  Yeah, 
keep  people  off  [on  their  toes]  their  feet. 
It  gets  boring  just  sitting  there  for  a few 
hours  at  a time  - off  balance,  give  them 
things  to  do  so  they  don't  get  bored. 

Karla:  It  [persistence]  is  motivating  you 
to  be  more  independent.  You  are  learn- 
ing how  to  be  more  independent 
There  are  games  they  can  play,  even 
with  adults  they  can  play  games.  Games 
to  help  our  minds  think  more  and  get 
that  energy  up  - motivate  our  brains. 
Give  us  ideas. 

[Have good  teachers?] 
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We  all  help  each 
other  sometimes. 
If  the  teacher  is 
somewhere  else 
we  just  ask  the 
others  students. 
Teamwork! 


Karla:  In  the  adult  literacy  programs 
there  are  good  teachers.  That  about  sums 
it  up  for  me.  They  know  what  they  are 
doing.  They  interview  the  right  people. 
[Do  they  ever  ask  the  students  opinions 
when  hiring  new  teachers?]  No,  some 
programs  do  and  they  should  do  it  at  all 
programs.  If  they  were  to  do  it  here. . . I 
think  I speak  for  all  of  us.  We  can  all  say 
that  we  have  good  teachers  here.  They 
make  us  feel  comfortable.  I don't  feel 
uneasy. 

[Is  talking  important?] 

Sam:  Communication! 

Karla:  The  teachers  talking  to  the  stu- 
dents individually  and  finding  out  what 
are  their  goals.  What  do  they  want  to  do 
and  what  they  are  there  for? 

They  communicate  with  the  students. 
You  are  not  there  like  a number.  We  are 
not  just  there  to  them.  The  government 
funds  it  and  they  are  doing  the  right 
thing. 

[A  student  once  told  me,  "this  is  our  turf'; 
students  can  tell  if  the  program  cares  - how 
can  you  tell?] 

Sam:  Well  they  just  "flip  you  off",  like 
[programs  that  don't  care].  It  is  like  if 
they  say  four  words  to  you  and  walk 
away  from  you  or  they  just  don't  care 
about  your  grade  They  give  you  an  (F) 
no  matter  what  you  do.  They  give  you 
whatever  they  want  to  give  you  even  if  it 
is  below  what  you  think  you  should 
have  got. 

[How  about  a pat  on  the  back  for  coming 
back  to  school?] 


Karla:  Yes,  for  coming  to  school. 

Sam:  You  are  supposed  to  come  to 
school  "anyways",  Karla? 

Karla:  Wait  a minute,  Sam!  They  should 
pat  on  the  back,  but  give  you  a little 
something,  not  an  award.  But  if  they 
have  perfect  attendance  or  something 
like  that,  a gift  certificate  or  something 
like  that  It  will  set  an  example  for  the 
ones  who  don't. 

Sam:  Maybe  the  ones  who  don't  come  in 
don't  want  to  be  there. 

Karla:  Not  necessarily,  Sam.  There  are 
thousands  of  problems  in  some  peoples 
lives.  Sometimes  there  are  things  like... 
health-wise,  then  there  are  children  and 
then  there  are  doctors  and  sickness. 
Somebody  could  be  taking  care  of  a 
mother  - running  errands  for  that  per- 
son. 

Sam:  Did  you  ever  hear  of  tutors? 

[Students  need  to  be  treated  like  everyone 
else] 

Sam:  They  should.  You  got  like,  you 
come  in  and  maybe  he  doesn't  know 
how  to  do  something  yet 

Karla:  We  all  help  each  other  sometimes. 
If  the  teacher  is  somewhere  else  we  just 
ask  the  others  students.  Teamwork! 

[Show  students  progress?] 

Karla:  We  have  goals  we  want  to... 
[achieve].  Here  they  don't  give  tests. 
Here  they  take  you  and  say  like...  Give 
you  some  pointers.  They  don't  [give 
tests].  They  would  if  you  asked.  I won- 
der if  they  need  to  do  that?  I would  like 
to  [have  a test]. 
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Sam:  Some  people  get  nervous.  My 
hands  use  to  shake.  They  tell  you,  you 
are  going  to  have  a test  this  Friday.  I just 
couldn't  do  it.  I'd  do  the  questions  and 
as  soon  as  the  test  finished  I forgot. 
Karla:  I got  frustrated  because  the  words 
just ....  they  look  so  much  alike  to  me  as 
an  adult.  I was  given  a lot  of  tests  that  I 
didn't  pass. 

[What  can  others  learners  do  to  help  there 
classmates  develop  persistence?] 

Sam:  See  what  the  program  is  all  about 

-11  j/L-j. 

CUIU  CUi  UUtL. 

Karla:  Tell  them  We  do!  We  tell  the 
students  about  the  program  - the  differ- 
ent areas  and  the  different  times.  There 
are  options,  there  is  math,  there  is 
reading  time.  Options  is  when  you  play 
games  and  usually  it  is  a learning  game. 

Well  yes,  the  students,  the  students 
are  nice,  real  nice  to  each  other. 

Sam:  They  don't  rank  on  anybody.  We 
are  grownups,  you  can't  do  that 

[ What  can  family  and  friends  do  to  help 
students  develop  persistence?] 

Sam:  They  can  give  you  less  grief.  Like  if 
you  are  married  or  have  kids  your 
spouse  will  take  care  of  them  while  you 
are  doing  your  homework. 

Karla:  It  is  going  to  cause  extra  work. 
That  means  you  are  going  to  have  to 
figure  out  when  to  cook  supper,  when  to 
set  that  table  and  have  supper. 


If  you  are  single  it  is  a matter  of  you 
making  time  for  yourself. 

Sam:  Let  the  kids  do  it. 

Karla:  Yeah,  teamwork!  Then  after  that 
you  have  got  time  to  study. 

Sam:  You  can  do  it  in  between  soap 
operas. 

Karla:  You  can  do  it  in-between  cooking 
too. 

[Karla,  before  you  talked  about  managing 
your  own  work.] 

Karla:  1.  Knowing  what  you  need,  2. 
Developing  strategies  to  get  their  goals. 

Sam:  3.  Choose  materials  to  help  you.  4. 
You  should  know  enough  about  yourself 
and  what  kind  of  learning  strategies  or 
the  different  ways  to  solve  problems  that 
work  best  for  you. 

Karla:  5.  Evaluating  your  own  learning. 

[Are  persistence  and  managing  your  own 
work  connected?] 

Sam:  Yes  they  are.  I think  so.  Sure.  Yeah. 
Like  um... 

Karla:  Like  getting  yourself  to  the 
program  - involving  yourself  in  the 
classes,  in  options  and  all  kinds  of 
things.  Finding  out  other  choices.  Just 
being  persistent  and  asking  the  teachers 
for  more  work  and  help. 
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CONCLUStON/UFUCTIONS 

I need  to  next  conduct  an  additional 
field-test(s),  based  partially  on  recom- 
mended changes  from  the  participants  in 
the  initial  field-test.  The  most  significant 
change  will  be  to  increase  the  size  of  the 
group,  modify  some  dyad  activities, 
using  some  of  the  critical  dialogues  from 
the  first  trial  run  as  readings  in  the  next 
field-test,  and  improve  the  group  editing 
process  in  developing  their  own  critical 
dialogues,  by  putting  an  emphasis  on  it 
being  more  of  a participatory  activity, 
rather  than  simply  a task  that  needs  to 
be  completed. 

Three  months  after  completing  each 
trial  run  I will  return  to  the  field-test  site 
in  order  to  determine  if  there  had  been 
any  continued  movement  on  the  part  of 
the  participants  towards  becoming  self- 
directed  and  whether  or  not  this  "stuff' 
stuck. 

WHAT  HAVK  I LEARNED  SINCE  CONDUCT! NO 
THE  FIRST  FIELD-TEST 

Prior  to  starting  the  program  I gave 
both  participants  a pre-test  I also  gave 
them  the  same  test  again  at  the  end  of 
the  program.  It  will  also  be  given  again 
three  months  after  the  end  of  the  pro- 
gram — along  with  an  interview  — and 
used  to  form  part  of  the  basis  of  an 
interview  guide. 

After  each  workshop  I listened  to  the 
tape  and  then  filled  out  an  observation 
sheet.  It  was  basically  a scaled  checklist 
of  indicators  (30)  that  I developed.  It  was 
mostly  based  on  the  intended  learning 
outcomes  of  the  program  and  one 
unintended  learning  outcome  that  I 
became  aware  of  after  the  second  com- 
ponent. 

To  be  brief,  Karla  initially  indicated 
that  she  blamed  herself  because  she  had 


to  return  to  school  as  an  adult,  was 
ashamed  to  be  back  in  school  as  an  adult 
and  was  incapable  of  doing  academic 
work  on  her  own.  According  to  her  post- 
test results,  Karla  indicated  that  she  has 
stopped  blaming  herself,  is  not  ashamed 
to  be  back  in  school  as  an  adult  and 
sometimes  feel  capable  of  doing  aca- 
demic work  on  her  own.  In  all  she  has 
developed  a more  positive  self-concept 
My  own  observations  were  in  agree- 
ment with  what  Karla  had  indicated  on 
the  post  test.  It  was  also  clear  that  she 
recognized  and  gave  herself  credit  for 
overcoming  many  barriers  in  returning 
to  school.  (The  pre-post  test  of  this 
particular  ILO  was  faulty  and  needs  to 
be  redesigned.) 

Sam  on  the  other  hand  had  finished 
high  school  and  did  not  have  as  negative 
a self-concept  of  himself  as  Karla  before 
starting  the  program.  There  was  still 
some  indication  of  positive  movement  in 
the  right  direction. 

My  own  observational  checklist 
indicated  a little  more.  But  for  Sam  the 
biggest  improvement  area  may  have 
come  about  as  a result  of  what  he 
learned  by  being  in  the  same  group  with 
Karla.  Developing  more  of  a sense  of 
critical  awareness  became  an  unintended 
learning  outcome  during  the  second 
component  of  the  program.  Over  the 
course  of  the  next  five  workshops  Sam 
appeared  to  be  developing  more  of  a 
sense  of  critical  awareness.  Some  of  the 
indicators  of  this  were  that  the  partici- 
pants were: 

1.  Verbally  critiquing  the  educational 
system 

2.  Breaking  through  apathy 

3.  Developing  an  awareness  of  the 
causes  of  problems 
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4.  Recognizing  that  change  comes  from 
the  bottom 

5.  Planning  to  or  getting  more  actively 
involved  in  activities  that 
propose  to  bring  about  change 

From  reading  the  critical  dialogues,  it 
was  apparent  that  Karla  had  a keen 
sense  of  critical  awareness.  Sam  was 
more  "laid  back"  and  willing  to  give  the 
system  time  to  work.  By  the  end  of  the 
program  he  was  developing  more 
critical  awareness. 

This  is  only  an  evaluation  research 
project.  I'm  certainly  not  going  to  at- 
tempt to  generalize  from  two  partici- 
pants. But  it  did  help  these  learners  and  I 
did  learn  from  them. 
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What  Counts? 

The  Right  Answer:  There  Is  More  Than  One 
Adnlt  Diploma  Program  Math  Research 


by 

Susan  Barnard 
and 

Kenneth 
Tamar  kin 


SCALE 

Somerville,  MA 


The  Adult  Diploma  Math  assess- 
ment project  was  a collaboration 
of  Kenneth  Tamarkin  and  Susan 
Barnard.  Kenny  is  a member  of  the  Math 
team  and  Susan  is  the  Program  Adminis- 
trator for  the  Adult  Diploma  Program 
(ADP)  at  SCALE.  We  wanted  to  update 
SCALE'S  ADP  Math  Assessment  to 
reflect  the  NCTM  standards,  specifically 
math  as  problem  solving. 

The  premise  of  the  ADP  is  to  earn  a 
high  school  diploma  by  demonstrating 
proficiencies  in  life-skills  based  compe- 
tencies that  have  components  of  reading, 
writing  and  math  interwoven  through- 
out the  curriculum.  In  order  to  start  the 
ADP  process,  students  must  have  an 
eighth  grade  reading  comprehension 
level  (Form  4B  of  Nelson)  if  a native 
English  speaker,  and  a 7th  grade  reading 
level  if  a non-native  speaker;  an  eighth 
grade  math  level;  and  the  ability  to  write 
a paragraph  with  90%  accuracy  in 
structure,  grammar  and  spelling. 

Over  the  years,  we  have  identified 
critical  thinking  as  the  weakest  skill  area. 
We  found  that  we  can  teach  to  the  ADP 
math  entrance  test  and  students  can 
quickly  pass.  But  given  a similar  calcula- 
tion, or  problem  out  of  context  or  with  a 
change  in  format,  students  are  not 
transferring  their  skills.  This  project  was 
an  opportunity  to  see  how  students 
would  respond  to  a questions  that  could 
have  more  than  one  answer.  After  all, 
isn't  that  what  life  is  all  about? 

In  our  plan  to  implement  the  project. 


we  targeted  students  who  attended 
ADP  intake/orientation  sessions  (during 
which  they  completed  an  ADP  math 
pre-test),  so  we  could  establish  some 
indication  of  the  person's  current  math- 
ematical abilities.  As  we  became  more 
involved  in  the  project,  however,  our 
target  group  expanded  to  include  an 
ABE  class,  and  ADP  preparation  class, 
a class  at  Quinsigamond  Community 
College,  and  a few  staff  members.  In 
total  we  involved  36  participants  in  our 
project  30  students;  6 staff.  The  profile 
of  the  students  varied  greatly  in  reading 
levels,  nationalities,  ages,  and  educa- 
tional backgrounds.  The  common  factors 
were  that  all  students  were  over  18  years 
of  age,  all  had  the  goal  of  achieving  a 
high  school  credential,  and  all  were 
motivated  to  attend  classes  without 
stipends. 

To  address  our  goal  of  including  more 
problem-solving  skills  in  the  ADP  math 
curriculum,  we  developed  a set  of  six 
open-ended  math  questions,  all  of  which 
had  a range  of  correct  answers.  In  order 
to  focus  on  problem  solving  rather  that 
computational  skills,  we  wrote  all  the 
questions  so  that  they  could  be  correctly 
answered  using  whole  numbers,  though 
some  participants  chose  to  use  decimals 
or  fractions  in  answering  some  prob- 
lems. The  directions  stated  that  a prob- 
lem could  have  more  than  one  answer, 
but  only  one  answer  was  required  to  be 
written  down.  Our  criteria  for  success 
were  to  have  questions  that  both  drew 
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the  participants  in  and  revealed  their 
thinking;  A follow  up  interview  sheet 
was  developed  to  gather  individual 
input  from  the  participants — including 
their  reaction  to  the  test  questions,  the 
test  format  and  the  test  purpose. 

We  hoped  to  find  out  whether  they 
considered  the  new  questions  fair  or 
unfair,  easy  or  hard,  and  clear  or  confus- 
ing. We  particularly  wanted  to  find  out 
if  the  participants  recognized  that  the 
new  questions  were  meant  to  challenge 
their  thinking  skills.  Each  participant 
was  individually  interviewed  and  the 
answers  recorded  on  the  interview  sheet 

The  chart  on  the  last  page  provides  an 
overview  of  our  results,  the  score  on  the 
ADP  math  pre-test,  and  the  new  ques- 
tions, along  with  a graph  displaying  the 
relationship  of  pre-test  and  new  ques- 
tions scores. 

To  understand  the  relevancy  between 
this  test  and  the  six  new  questions  we 
asked,  it  is  helpful  to  understand  the 
scoring.  In  the  pre-test,  the  highest 
possible  score  is  28.  If  students  score 
between  18-28,  a quick  1-3  hour  math 
review  usually  brings  their  math  abilities 
to  the  level  where  they  can  pass  the 
math  entrance  diagnostic;  a score  of  10- 
17  visually  requires  an  ABE  class  from  3 
months  to  a year. 

On  our  six  math  questions,  the  highest 
score  could  be  6,  with  credit  given  for 
partially  correct  answers.  As  we  began 
scoring  the  six  questions,  we  found  it 
necessary  that  one  person  do  the  scoring 
in  order  to  achieve  consistency.  Other- 


espedally  for  partial  credit 
When  asking  people  to  participate,  we 
said  we  were  thinking  about  revising  the 
ADP  math  curriculum  and  needed  their 
help  and  input  None  of  the  test  sessions 
were  times;  students  were  assured  there 
was  no  penalty  for  guessing.  The  partici- 
pants' willingness  to  work  with  us  and 
give  us  honest  and  complete  feedback 
was  outstanding.  Word  of  the  project 
spread  and  we  had  teachers  asking  if 
they  could  try  the  questions  with  their 
students.  ( Those  results  are  not  included 
in  our  findings.) 

Wp  decided  to  write  up  the  results  of 
our  findings  separately  and  then  met  to 
discuss  our  findings.  To  our  surprise, 
our  write  ups  were  strikingly  similar. 

QUESTION  1:  You  have  $100  to 
share  among  three  people.  You  don't 
have  to  divide  the  money  equally,  but 
no  person  can  have  less  than  $25 . How 
much  does  each  person  have? 

We  intentionally  picked  numbers  that 
couldn't  be  divided  evenly  and  stated 
that  the  three  people  did  not  have  to 
get  the  same  amount  Yet,  we  found 
that  most  participants  wanted  to  be 
fair  by  dividing  the  money  equally  in 
three  ways,  giving  each  person  $33,  or 
even  $33.33,  often  losing  sight  that  the  > 
amounts;  once  divided,  should  add  up 
to  $100.  One  participant,  while  being 
interviewed,  commented, " of  course 
we  divided  everything  evenly.  We're 
parents. 


wise,  with  more  than  one  answer  pos- 
sible and  partial  credit  given,  each 
scorer's  bias  could  distort  the  scores. 
As  a follow  up  task  to  this  research,  we 
need  to  standardize  scoring  criteria. 


QUESTION  2 : You  go  to  the  grocery 
store  with  $25  in  our  pocket  to  buy 
milk,  orange  juice,  bread  and  shampoo. 
What  other  items  would  you  buy  so 
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that  you  would  go  home  with  no  less 
that  $10.00  in  your  pocket?  How  much 
money  would  you  have  left? 


bread 

Grocery  items: 
$1  cookies 

$2 

spaghetti 

$1 

tooth  paste 

$2 

cat  food 

$1 

cucumbers 

$ 1 

cheese 

$4 

milk 

$2 

pancake  mix 

$1 

orange  juice 

$2 

fish 

$4 

ice  cream 

$3 

lettuce 

$1 

lemons 

$1 

shampoo 

$4 

People  seemed  to  particularly  enjoy 
this  question,  probably  because  people 
love  to  shop.  If  they  bought  only  the 
required  items,  they  would  have  $16  left, 
which  was  a correct  answer.  Yet,  people 
seemed  to  focus  on  the  $10,  opting  to 
buy  more,  going  home  with  exactly  that 
amount.  Other  people  did  not  organize 
or  label  their  work  and  became  con- 
fused, sometimes  thinking  $10  more 
needed  to  be  spent  and  forgetting  they 
needed  to  go  home  with  $10.  Overall, 
this  question  had  the  highest  rate  of 
correct  answers. 

Question  #3:  We  showed  them  two  jars 
and  said:  There  are  10  pieces  of  candy  in 
one  jar.  Estimate  the  number  of  pieces  of 
candy  in  the  other  jar. 

The  other  jar  contained  105  pieces  of 
candy  wrapped  in  cellophane.  The 
idea  was  to  see  if  people's  answers 
would  be  in  the  ballpark.  We  decided 
a range  of  30  pieces  in  either  direction 
would  be  an  accepted  answer 
(because  Kenny  guessed  130  and 
didn't  want  to  be  wrong.)  A surpris- 
ing number  of  people  got  this  answer 


wrong;  quite  a few  answered  70.  The 
problem  was  the  paper  added  extra 
volume  and  the  candy  could  be 
tightly  or  loosely  packed,  making  it 
difficult  to  estimate  accurately. 
Participants  who  guessed  50  or  below 
also  had  severe  problems  with 
computation  and  seemed  to  lack  basic 
number  sense.  We  would  alter  this 
question  if  we  make  it  a permanent 
part  of  ADP  assessment.  We  would 
use  M&Ms,  beans  or  some  other 
unwrapped,  regularly  shaped,  small 
object  as  the  standard  so  that  the 
volume  estimation  could  be  more 
consistent. 

Question  #4:  Draw  a rectangle  whose 
perimeter  is  8 inches. 

In  this  question,  participants  had  to 
know  what  a rectangle  and  perimeter 
are.  The  standard  approach  to  this 
question  is:  What  is  die  perimeter  of  a 
rectangle  whose  sides  are  3, 1, 3, 1? 
The  question  also  emerged,  "Is  a 
square  a rectangle?  Yes,  it  is.  There 
were  two  whole  number  correct 
answers:  A square  with  2 inch  sides 
and  a 3 inch  by  1 inch  rectangle.  Most 
people  drew  the  3 by  1 rectangle.  We 
also  accepted  as  correct  a 2 1 /2  by  1- 
1/2  inch  rectangle,  since  we  did  not 
require  that  the  answer  had  to  be  in 
whole  numbers.  One-third  credit  was 
deducted  if  the  lengths  of  the  sides  of 
the  rectangle  were  not  labeled.  If  we 
implemented  this  question,  we  would 
directly  ask  for  the  sides  to  be  labeled, 
since  this  requirement  was  not 
entirely  clear.  A number  of  people  did 
not  even  attempt  this  question, 
considering  it  too  hard.  Others  drew  8 
by  1 inch  or  2 by  4 inch  rectangles. 
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mistaking  area  for  perimeter. 

Question  #5:  A room  is  15*  by  12'.  You  need 
to  place  a wood  burning  stove  that  is  3'  x 2' 
in  the  room.  The  stove  must  be  at  least  V 
from  any  wall.  Draw  a floor  plan  that  shows 
where  the  stove  can  be  located. 

This  question  encouraged  more 
creativity.  Although  there  were  many 
correct  answers  to  this  question,  many 
participants  did  not  even  attempt  it, 
again  stating  that  the  question  was  too 
hard.  Some  people  did  draw  good 
floor  plans,  but  did  not  label  the 
duTicTiSiortS/  missing  the  importaiiCc 
of  clearly  communicating  their 
conclusions.  During  the  interview, 
participants  said  they  wondered  if  the 
plan  should  be  drawn  to  scale  or  if 
they  should  label  things.  We  would 
alter  the  question  to  make  clear  that 
the  dimensions  and  distances  need  to 
be  written  on  the  diagram. 

Question  #6: 250  - 74  + 120  = 296 . Using 
these  numbers,  write  a word  problem. 

This  question  also  required  a bit  more 
creativity  and  the  answers  were  fun  to 
read.  Some  were  quite  entertaining 
and  creative,  as  well  as  being  correct, 
but  many  people  had  difficulty  writ- 
ing an  accurate  word  problem.  Partici- 
pants had  the  most  difficulty  phrasing 
a question  that  would  lead  someone 
to  write  an  answer.  Instead,  many 
included  the  answer  in  their  word 
problem.  For  example,  one  word 
problem  said,  "I  have  250  $ in  the 
Bank  I took  74  from  it  to  pay  my  bill, 
and  the  next  week  I deposit  120  now 
my  balance  from  the  bank  is  2 96  $." 
We  feel  that  clarity  in  the  directions  is 
needed.  Something  like:  The  answer 


should  not  be  stated  in  your  word 

problem. 

On  our  math  survey  we  asked 
participants  to  rate  the  problems  from  1 
to  5 in  the  following  categories:  Fair  to 
Unfair,  Easy  to  Hard  and  Gear  to 
Confusing.  Most  students  thought  that 
the  questions  were  Fair  and  Gear, 
responding  to  these  questions  with  a 1 or 
a 2.  The  Easy  to  Hard  question  mostly 
received  3's. 

During  the  interviews,  people  who 
did  best  on  the  traditional  test  and  very 
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particularly  enthusiastic.  Some 
comments  were  "It  makes  you  think", 

"It  was  fun."  The  few  negative  reactions 
came  mostly  from  people  who  had  done 
poorly  on  both  the  traditional  test  and 
the  experimental  questions.  They  had 
comments  such  as  "it  was  confusing," 
and  a number  of  these  poor  math 
achievers  were  unsure  exactly  why  we 
were  trying  to  do  this  project.  However, 
overall,  we  were  pleased  to  find  out  that 
almost  everybody  was  very 
cooperative  and  actually  enjoyed  being 
asked  for  their  input  before  changes 
were  implemented. 

In  analyzing  our  statistics,  we  found 
that  of  the  14  participants  who  got  1 /2  or 
less  of  the  ADP  pretest  correct  (14  or 
below),  only  4 got  1 /2  (3  or  more) 
correct  on  the  new  questions;  only  one 
got  4 out  of  6 correct.  Of  those  16 
participants  who  got  more  than  1/2 
correct  on  the  ADP  pretest  (15  to  28), 
only  2 scored  below  1/2  correct  (less 
than  3)  on  the  new  questions. 

The  results  raise  some  interesting 
questions.  Is  there  a significant 
correlation  between  computational  skills 
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and  reasoning  skills?  Can  someone  who 
was  unsuccessful  learning  math  in  a 
computation-oriented  program  learn 
more  effectively  in  a problem-solving 
oriented  program? 

The  open-ended  questions  were 
definitely  more  interesting  to  score.  The 
variety  of  answers  and  approaches  gave 
great  insight  into  how  the  student  was 
thinking.  The  wrong  answers  were 
particularly  helpful  in  showing  how  an 
individual  approached  a problem,  and 
also  when  the  working  of  the  problem 
was  open  to  misinterpretation.  All  our 
questions  met  our  criteria;  we  just  need 
to  refine  and  revise  our  directions. 

Our  conclusion  is  that  we  should 
proceed  with  altering  the  ADP  tests  and 
curriculum  to  include  open-ended 


questions  and  problem-solving  skills. 

We  believe  this  will  create  a richer,  more 
relevant  experience  that  will  translate 
well  into  improved  functioning  and 
critical  thinking  in  everyday  life.  We  also 
thought  that  it  was  important  to  give 
people  options  in  the  testing  situation, 
since  that  more  closely  reflects  real  life 
than  exclusively  having  questions  with 
only  one  correct  answer.  The  correlation 
that  we  observed  between  achievement 
on  the  computation-oriented  exam  and 
the  problem  solving  oriented  questions 
also  suggests  to  us  that  there  is  a role  for 
effective  computational  instruction  and 
that  progress  in  problem-solving  skills 
can  indeed  be  meaningfully  measured 
and  evaluated. 


Reprinted  from  the  Mass  ABE  Standards,  with  permission . 
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Working  with  Industry 

Authentic  Assessment  in  the  Workplace 


C&K  Components  is  a switch 
manufacturing  company 
based  in  Newton,  MA.  The 
company  has  had  an  ESL  program  for 
approximately  five  years.  For  an  18 
month  period  from  October,  1992 
through  December,  1993  (three  20-week 
cycles)  the  program  was  funded  by  a 
federal  grant.  The  program  always 
included  two  classes  running  simulta- 
neously, although  for  a 10-week  period 
in  the  fall  of  1993,  four  classes  ran  simul- 
taneously. I was  the  Site  Coordinator/ 
Instructor  at  C&K  during  the  18  month 
grant  period,  and  for  an  additional  20- 
week  cycle  after  the  grant  period  ended, 
teaching  two  classes  each  cycle.  Another 
instructor  (Elinor  Pitkin)  taught  the 
additional  two  classes  in  1993;  these 
classes  were  added  because  there  was 
money  left  in  the  budget.  It  was  discov- 
ered though  that  production  could  not 
handle  having  that  many  workers  off  the 
line  at  the  same  time,  so  after  that  we 
returned  to  offering  only  two  classes. 
Most  of  the  participants  in  the  program 
work  as  assemblers  and  machine  opera- 
tors, although  there  have  also  been  some 
stockroom  clerks  and  line  and  unit  leads. 

We  (the  instructors  and  Workplace 
Education  Coordinator  at  JVS)  used  an 
initial  assessment  tool  for  the  purpose  of 
placing  students  in  one  class  or  the  other 
based  on  their  abilities  and  levels.  For 
the  first  cycle  of  classes,  the  levels  were 
predetermined  based  on  the  needs  and 
wants  of  management:  one  class  was  for 


those  who  held  semi-supervisory  posi- 
tions (line  and  unit  leads),  assumed  to  be 
advanced.  The  other  was  for  beginners. 
Management  wanted  to  help  those  in 
semi-supervisory  positions,  in  particular, 
those  individuals  they  felt  would  be  able 
to  offer  something  to  the  company.  In 
particular  management  hoped  that  by 
helping  leads  ieam  work-related  topics, 
the  leads  in  turn  would  be  able  to  train 
their  line  workers.  For  example,  manage- 
ment wanted  the  instructor  to  teach 
leads  how  to  read  and  understand  work 
orders  so  that  they  could  turn  around 
and  teach  this  skill  to  others. 

Management  had  the  same  goal  for 
assemblers  in  the  lower  level  class:  that 
they  would  learn  work-related  skills  and 
then  teach  them  to  their  co-workers  on 
the  line.  They  also  wanted  to  groom 
those  beginners  they  felt  had  a chance  of 
advancing  to  lead  positions.  They  there- 
fore hand-picked  "strong"  employees  for 
us  to  assess  and  place  in  class. 

The  original  assessment  tool  we  used 
at  the  start  of  the  grant  period  consisted 
of  two  components:  an  oral  interview 
and  a writing  sample.  The  oral  interview 
consisted  of  ten  questions  which  at- 
tempted to  elicit  both  factual  and  infer- 
ence information,  and  the  ability  to  use 
different  tenses.  The  interview  was 
scored  by  checking  off  whether  the 
interviewee  answered  at  all,  answered 
appropriately  or  not,  and  whether  the 
answer  was  complete  and  grammatical. 
Some  interviewers  also  attempted  to 
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write  down  the  interviewees'  responses, 
but  others  did  not  do  so.  The  inference 
questions  addressed  a photograph,  and 
then  shifted  from  the  photograph  to  the 
students'  own  life  dreams.  The  photo- 
graph was  not  a picture  that  had  any 
connection  to  the  lives  of  the  employees; 
one  of  the  first  changes  we  made  in  the 
assessment  was  to  use  a picture  of  an 
assembly  worker.  Employees  suddenly 
had  much  more  to  say  to  the  inter- 
viewer. 

The  writing  component  gave  employ- 
ees a choice  of  two  topics  to  write  about, 
and  asked  them  to  write  a paragraph. 
They  were  also  asked  (in  writing)  why 
they  wanted  to  study  English. 

At  the  start  of  the  program,  we  per- 
ceived the  assessment  test  in  very  nar- 
row terms,  considering  only  the 
interview  and  writing  components  once 
potential  students  arrived  in  the  room 
for  testing.  Over  the  course  of  several 
months  (2-3  cycles),  a number  of  issues 
arose  which  caused  us  to  reconsider  the 
initial  assessment.  These  issues  included 
the  following: 

1.  Multi-level  classes.  Since  we  only 
tested  a few  more  students  than  could 
be  placed  in  classes,  we  were  in  the 
position  of  filling  classes  with  who- 
ever we  tested,  regardless  of  actual 
level  of  ability.  While  trying  to  keep 
the  classes  as  homogeneous  as  pos- 
sible, we  still  ended  up  with  a wide 
range,  especially  at  the  lower  levels, 
where  we  would  have  literacy  and 
beginner /high  beginner  students  in 
the  same  class.  Part  of  the  problem 
was  that  the  test  did  not  accurately 
reveal  skills,  particularly  at  the  lit- 
eracy and  beginner  levels.  Both  teach- 


ers and  students  were  concerned 
about  the  level  differences  within 
classes.  It  is  important  to  emphasize 
the  students'  role  in  prompting  us  to 
review  the  assessment  tool;  they 
voiced  strong  concerns  over  the  mixed 
level  classes  and  over  low  motivation 
of  some  students.  Responding  to  these 
concerns  involved  re-thinking  the 
assessment.  The  other  teacher  also 
played  an  important  role;  coming  in 
as  an  outsider  for  a short  period  of 
time,  she  was  able  to  pinpoint  issues 
and  concerns  which  I had  felt  but  not 
yet  articulated. 

2.  Initial  selection  process.  We  began  to 
question  the  company's  recruitment 
policy  for  a couple  of  reasons.  We 
tried  offering  more  classes  (adding 
two  in  the  fall  of  1993),  but  manage- 
ment found  that  when  we  ran  four 
classes,  there  were  too  many  people 
off  the  line.  Production  could  not 
handle  this  need  in  terms  of  numbers. 
We  also  wanted  to  test  many  more 
people  than  we  could  serve  as  a way 
to  resolve  the  problem  of  the  multi- 
level classes.  However,  management 
opposed  this  idea  because  they  did 
not  want  to  raise  false  expectations  by 
testing  employees  when  they  could 
not  serve  them;  management  felt  that 
open  testing  would  result  in  too  many 
people  being  tested  and  too  much 
disappointment.  We  also  felt  that 
supervisors  and  managers  were  pre- 
selecting those  employees  they  felt 
would  further  benefit  the  company 
through  classes,  and  ignoring  other, 
potentially  successful,  employees.  It 
seemed  that  they  were  selecting  those 
who  were  bound  to  do  well.  They 
were  also  pre-selecting  based  on  their 


fURES  IN  ASSESSMENT 


574 


Volute  7:  Fall  1994 


The 

Partnenhlp  81 
Project 


idea  of  who  was  'advanced'  and  who 
was  'beginner',  but  their  perceptions 
did  not  necessarily  match  skill  levels 
reflected  on  an  assessment  test  Sec- 
ond, and  related  to  the  pre-selection 
process,  was  confusion  and  resent- 
ment on  the  part  of  employees  over 
who  was  chosen  to  be  in  the  program; 
some  employees  were  able  to  partici- 
pate more  than  once  while  others 
never  even  got  tested  even  though 
they  wanted  to  study. 

3.  Curriculum  and  materials  develop- 
ment We  found  that  the  original 
initial  assessment  did  not  provide 
instructors  with  the  information  they 
needed  for  determining  what  to  teach, 
and  at  what  level  The  assessment  did 
not  provide  students  an  opportunity 
to  provide  input  into  shaping  the 
curriculum.  Nor  were  instructors  sure 
of  the  best  way  to  approach  the 
resulting  multi-level  classes. 

As  a result  of  the  concents  of  instruc- 
tors and  students,  and  with  the  intention 
of  improving  service  to  the  company 
and  students,  during  the  third  cycle  of 
classes  we  (the  instructors  and  coordina- 
tor) began  to  re-look  at  the  assessment 
tool  and  ultimately  re-design  it  The  first 
concern  we  needed  to  address  was  what 
the  assessment  tool  was  for  in  the  first 
place:  what  information  we  wanted  to 
get  from  the  assessment,  what  skills  we 
wanted  to  assess,  who  we  were  assess- 
ing for,  and  how  that  information  would 
be  used. 

We  decided  that  there  were  a number 
of  reasons  for  assessment 

1.  determine  level  of  motivation  and 
interest  in  classes 

2.  place  in  appropriate  and  homoge- 
neous classes 


3.  establish  and  standardize  proficiency 
levels 

4.  measure  learner  progress  over  time 

5.  track  program  effectiveness  (by 
showing  learner  progress)  (We  have 
since  re-thought  this  point  because  learner 
progress  is  not  the  only  - and  probably  not 
the  most  important  - measure  of  program 

effectiveness.) 

6.  ascertain  and  clarify  needs  and  expec- 
tations of  both  participants  and 
teacher. 

We  saw  a need  for  some  mechanism 
that  went  beyond  a pen  and  paper  test, 
that  would  ensure  an  ability  to  shape 
classroom  instruction,  and  that  would 
provide  potential  participants  with 
enough  knowledge  to  choose  whether  or 
not  they  wanted  to  take  a class.  The  new 
assessment  tool  intended  to  address 
these  reasons  or  issues,  consists  of  five 
components:  an  orientation  for  potential 
students,  an  interview,  a reading  section, 
a writing  section,  and  a literacy  assess- 
ment 

The  purpose  of  the  orientation  is  to 
provide  information  to  potential  stu- 
dents about  the  ESL  program,  what  is 
covered  in  classes,  and  what  the  expecta- 
tions are  for  participants,  and  how 
people  are  selected.  It  offers  potential 
students  the  opportunity  to  ask  ques- 
tions and  express  their  expectations, 
needs,  interest,  and  commitment.  This 
latter  point  was  particularly  important  to 
students  who  were  committed.  The 
orientation  gives  them  a chance  to 
choose  whether  to  participate.  In  retro- 
spect, we  see  that  many  people  who 
were  chosen  by  their  supervisors  may 
have  felt  obligated  to  attend  the  class, 
even  if  they  did  not  want  to.  This  lack  of 
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interest  and  motivation  is  reflected  in 
lack  of  completion  of  the  course,  poor 
attendance,  and/or  poor  attitude. 

The  interview  was  re-designed  in  two 
ways.  First,  we  re-did  the  way  responses 
are  scored,  to  provide  more  accurate 
information.  Second,  we  changed  some 
of  the  questions  so  that  they  would 
provide  more  useful  information  regard- 
ing abilities  and  needs.  Questions  are 
familiar,  in  that  they  ask  for  personal 
factual  information  and  about  the 
interviewees  own  life  goals. 

Originally,  the  assessment  did  not 
include  a reading  component,  but  we 
decided  to  add  one  because  we  felt  we 
needed  information  on  general  reading 
fluency,  decoding  skills,  and  reading 
comprehension.  We  decided  to  offer 
students  a wide  choice  of  reading  mate- 
rials, at  different  levels,  and  let  them 
choose  for  themselves  which  one  to  read. 
In  this  way,  they  could  show  us  what 
they  found  interesting  or  relevant  We 
found  that  students  read  better  when 
they  read  something  familiar,  either  in  a 
familiar  format  (e.g.  want  ads,  work 
memos),  or  a familiar  topic  (e.g.  immi- 
grant experiences,  work  stories).  Read- 
ing selections  included  work-related 
materials,  real-life  materials,  and  selec- 
tions from  readers.  We  rated  the  read- 
ings for  level,  based  on  difficulty  of  the 
grammar,  length  of  sentences,  and 
vocabulary.  Comprehension  questions 
include  yes/no,  information,  and  infer- 
ence questions;  we  also  note  students' 
answers,  pronunciation  problems,  and 
whether  the  selection  seems  too  easy  or 
difficult  The  idea  for  organizing  a 
reading  component  in  this  way  came 
from  an  assessment  developed  at 
Quinsigamond  Community  College  in 


Worcester,  Mass. 

The  writing  section  also  became  more 
detailed.  We  found  that  with  the  original 
test,  students  often  did  not  write  because 
they  could  not  think  of  anything  to  say, 
not  because  they  lacked  writing  ability. 
We  instruct  the  student  to  choose  a 
topic,  and  if  it  appears  that  they  are 
having  difficulty,  we  provide  them  with 
a card  with  specific  questions  about  the 
topic. 

If  it  appears  that  the  student  cannot 
write  at  all  (or  if  the  students  say  that 
they  cannot  write),  we  use  the  literacy 
assessment.  This  was  an  important  addi- 
tion to  the  assessment  tool  because 
initially  we  had  no  way  to  evaluate  the 
skills  of  someone  who  claimed  to  be 
unable  to  write,  and  many  of  the  stu- 
dents at  C&K  are  at  literacy  or  very  low 
levels.  The  literacy  assessment  provides 
information  on  letter  and  sound  identifi- 
cation, numeracy,  sight  word  recogni- 
tion, and  writing  of  personal 
information.  If  the  student  is  able  to 
decode  the  sight  words  well,  we  have 
them  attempt  the  simplest  level  one 
reading  selection. 

The  detailed  information  which  this 
expanded  assessment  tool  provides 
enables  an  instructor  to  look  back  at  the 
test  and  evaluate  skill  levels.  Guidelines 
for  the  assessment  result  in  greater 
consistency  in  scoring  among  different 
testers.  In  this  way,  if  the  teacher  is  not 
the  same  person  who  gave  the  test,  he  or 
she  is  still  able  to  compare  tests  and 
more  accurately  group  people  by  level, 
as  well  as  glean  significant  information 
for  pre-planning  and  developing  appro- 
priate materials  for  classroom  instruc- 
tion. 

Parts  of  the  revised  assessment  tool 
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were  piloted  for  cyde  3 in  the  spring  of 
1994;  specifically,  we  had  the  orienta- 
tion, and  we  used  the  literacy  and 
reading  components.  The  interview  and 
writing  components  had  not  been 
completely  revised  in  time  and  were  not 
piloted.  We  found  that  management  was 
still  reluctant  to  have  open  testing  for  all 
interested  employees,  but  we  were  able 
to  continue  testing  until  we  had  filled 
two  classes  with  fairly  homogeneous 
skill  levels.  While  the  revised  assessment 
takes  longer,  it  provides  more  accurate 
information. 

We  found  that  the  orientation  clarified 
goals  and  expectations  so  that  students 
were  aware  of  what  the  program  would 
offer.  It  brought  to  our  attention  what 
workers  wanted  the  program  to  offer. 
Some  workers  who  came  to  the  orienta- 
tion decided  not  to  take  the  dass.  It  also 
raised  other  issues,  such  as  possibly 
needing  to  revise  the  program  to  address 
particular  needs,  offering  specialized 
courses  for  students.  This  issue  has  not 
yet  been  addressed. 

The  literacy  component  provided 


much  needed  information  at  the  lower 
levels  and  enabled  us  to  better  place 
students  in  classes  and  use  appropriate 
materials. 

At  this  time,  we  have  neither  used  the 
complete  test  for  initial  assessment  of 
students,  nor  have  we  used  it  to  measure 
student  progress  or  program  effective- 
ness. We  plan  to  pilot  the  complete 
assessment  tool  and  see  if  it  adequately 
addresses  all  five  of  our  goals  during  the 
next  cycle  at  C&K  Components.  The 
original  assessment  did  not  address 
student  concerns  or  provide  us  with  the 
results  and  information  we  wanted  and 
needed.  So  far,  it  seems  that  the  revised 
version  makes  more  sense  in  terms  of 
assessing  abilities  and  informing  instruc- 
tion. However,  we  are  still  left  with  the 
apparent  contradiction  of  needing  to  test 
sufficient  number  of  workers  so  as  to 
create  homogeneous  classes,  but  not 
testing  so  many  that  we  raise  expecta- 
tions of  people  we  cannot  serve.  There- 
fore, a question  which  remains  for  us  is 
how  to  find  a middle  ground  that  will 
satisfy  all  parties. 
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1 . What  is  your  name? 

2.  How  do  you  spell  it? 

3.  Where  are  you  from? 

4.  What  is  your  job  now? 

5.  Did  you  work  in  your  country? 

6.  What  job  did  you  have? 

7.  When  did  you  come  to  the  U.S.? 

8.  How  many  years  have  you  been  working  at  C&K? 

9.  What  is  your  first  language? 
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CAR  INITIAL  ASSESSMENT 
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<8*  / 
/ // 

/ 

# 

? £ 
*/ 

1 0.  Why  do  you  wont  to  learn  English? 

• 

1 1 . What  do  you  need  English  for? 

1 2.  What  did  you  do  yesterday? 

1 3.  What  do  you  do  everyday? 

1 4.  What  will  you  do  this  weekend? 

1 5.  Do  you  speak  English  at  Work? 
Home? 

with  friends? 
with  supervisors? 

1 6.  Did  you  study  English  before?  Where? 

Additional  Comments 
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LITERACY  ASSESSMENT 

Letter  Identification  (use  index  cards  for  letter  identification,  numeracy  and  sight  words.  Mark  responses 
on  this  sheet) 


p b s z 

j c w h 

ch  sh  th  qu 

t d f v 

r 1 y x 

m n g k 

a e i o 

Numeracy: 

1 5 13  26  151  100  10  1993  2 4327 

Sight  Words: 

Mrs.  one  two  who  work  you  the  on  has  come  exit 
country  danger  school  live  mother  women 
father  people  where  heard  busy  men 

nobody  example  trouble  cough  debt  caution 
bouquet  straight  iron  Wednesday  sure  warning 
Writing: 

Name 

Address: 

number  street 

city  state  zip  code 

Adapted  from  literacy  assessment  by  Jon  Phillips,  International  Institute  of  Boston 
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The  following  is  a short  description  of  my 
experiences  as  a GED  recipient.  Through  it, 

I advocate  for  a non-traditional  form  of 
assessment  to  be  used  as  a way  of  evaluating 
my  potential  or  capability  as  a college 
student  and  member  of  society.  1 wanted  to 
■write  about  the  choices  I've  made  in  the 
hopes  that  educators,  employers,  and  peers 
will  understand  ihai  sometimes  no  standar- 
dized tests  can  judge  a person's  ability,  and 
that  above  all,  personal  or  life  experience 
should  be  valued. 

Early  in  1986, 1 made  a decision 
that  changed  my  life;  I stopped 
going  to  high  school  I have  to 
admit  that  it  wasn't  a very  tough 
decision  because  I thought  I was  doing 
theright  thing  for  me.  I was  the 
youngest  of  nine  children  and  the  fourth 
high  school  dropout.  All  my  brothers 
and  sisters  were  content  with  their  lives 
and  the  jobs  they  had.  Because  of  this,  I 
believed  the  key  to  a happy  life  was  to 
find  a "good"  job. 

I filled  out  job  applications  at  every 
office  in  town,  but  had  great  difficulty 
getting  interviews.  My  brothers  and 
sisters  grew  up  in  a different  time,  a time 
when  there  wasn't  much  competition  for 
entry  level  work. 

I re-evaluated  things.  I came  to  believe 
that  the  key  to  a happy  life  was  a 
combination  of  many  things  held 
together  by  education.  I decided  to  get 
my  GED  (General  Educational 
Development).  I went  to  Action  for 


Boston  Community  Development 
(ABCD)  and  signed  up  for  classes.  I was 
given  a placement  test  and  put  in  a high 
level  class.  Six  months  after  entering  the 
GED  program,  I took  the  five  tests  and 
passed. 

Upon  receiving  my  GED  in  1987, 1 
found  an  entry  level  job  in  the  business 
world  as  a data  entry  operator.  I was 
grateful  for  the  opportunity  to  work, 
learn,  and  better  myself,  but  was 
disturbed  by  the  misconceptions  of  me 
as  a GED  recipient.  Some  people 
stereotyped  me  as  not  being  bright  or 
taking  the  easy  way  out  because  they  felt 
the  GED  tests  weren't  as  challenging  as 
going  through  traditional  high  schooL 
Or  they  had  preconceived  ideas  of  how 
someone  who  left  high  school  would  be. 
These  experiences  made  me  work  twice 
as  hard  to  get  taken  seriously.  I felt  I had 
something  to  prove  because  I knew  I 
could  do  the  job. 

I was  given  a promotion  to  workflow 
administrator  after  six  months.  I wasn't 
thrilled  with  the  work  I was  doing,  but  I 
didn't  have  any  other  skills  than  what  I 
learned  on  the  job.  I wanted  and  needed 
more  education  to  get  further  ahead  in 
my  life.  Emerson  College  in  Boston 
offered  a good  program  so  I made  a trip 
to  the  school  for  an  application.  When  I 
started  to  fill  out  the  application,  I 
couldn't  complete  the  section  that  asked 
about  your  high  school  scores  and 
grades.  I wasn't  sure  what  to  show  the 
school  to  prove  I was  capable  of  attend- 
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ing.  I went  back  to  the  admissions  office 
and  had  a discussion  with  one  of  the 
counselors.  I was  told  that  since  I didn't 
have  high  school  transcripts  or  SAT 
scores  for  them  to  draw  a conclusion 
about  my  potential  they  wouldn't  accept 
me  to  the  school.  I had  to  show  them 
that  I was  able  to  do  college  level 
material 

I ended  up  taking  two  courses  at 
Emerson  without  applying  as  a student. 
The  thought  was  that  if  I did  well,  I 
could  use  these  grades  as  transcripts.  I 
did  well  in  both  courses  and  took  two 
more  before  officially  applying  to  the 
school  I also  did  well  in  the  second  set 
of  courses.  The  following  semester  I 
made  an  appointment  to  see  the  same 
guidance  counselor.  I brought  a portfolio 
of  all  of  my  class  assignments  from  the 
four  classes.  We  began  talking  and  he 
looked  over  my  grades  from  each  of  my 
classes.  I pulled  out  my  portfolio  of 
papers,  teste,  projects,  even  work  that  I 
did  during  the  day  for  him  to  review 
and  he  was  impressed.  What  got  me 


accepted  to  the  school  above  all  was  a 
40-page  draft  of  a screenplay  that  I did 
in  a senior  level  class  that  was  graded  A- 
by  a tough  professor. 

I still  attend  school  part-time  in  the 
evenings.  It  has  been  five  years  and  I still 
have  a few  years  left  to  go  before  I 
receive  my  Bachelor  of  Fine  Arte  degree 
in  Writing,  Literature,  and  Publishing. 
No  matter,  I know  I will  finish.  I have 
talked  to  other  GED  recipients  who  have 
said  they  stopped  telling  people  they  are 
GED  graduates  because  it's  easier  to  say 
T graduated  from  such  and  such  high 
school'  I think  about  this  all  the  time.  I 
always  want  to  tell  people  that  I am  a 
GED  recipient.  Mainly  because  I am 
proud,  but  partly  because  I want  to  help 
change  someone's  attitude  about  what 
the  people  who  receive  GEDs  are  able  to 
accomplish.  I hope  by  writing  my  story 
I can  share  my  belief  that  the  GED  is  not 
a guarantee  of  anything.  It  is  just  the 
foundation  you  need  to  keep  your 
options  open  and  your  choices  broad. 
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Letter 

A Response  to  Hofer  and  Larson  on  links  between 
literacy  Practices  & Community  Development 


Judy  Hofer  and  Pat  Larson  identified 
a critical  component  of  literacy 
work  in  their  article  "Reflecting  on 
the  links  between  literacy  practices 
and  community  development"  (AiA, 

VoL  5,  Fall,  1993).  Although  many 
literacy  practitioners  assume  that  con- 
nections exist  between  classroom  learn- 
ing and  community  action,  we  seldom 
name  these  connections  explicitly,  nor 
do  we  always  know  whether  the  learn- 
ers with  whom  we  work  share  those 
assumptions.  Do  we  believe  that  only 
through  living  and  working  within  a 
community  do  we  have  the  right  and  the 
opportunity  to  work  with  others  in  that 
community  towards  creating  positive 
social  change?  Is  our  task  to  help  adults 
learn  about  existing  options  within  their 
communities,  make  informed  choices 
and/or  create  new  possibilities?  Defin- 
ing what  we  mean  by  community  devel- 
opment may  be  slippery;  finding  ways 
to  assess  its  growth  is  an  ongoing  chal- 
lenge. 

In  North  America,  our  assumptions 
about  growth  and  learning  are  informed 
essentially  by  'first  world'  thinking.  This 
makes  sense;  contexts  in  which  adult 
education  occurs  in  Canada  are  not  the 
same  as  those  occurring  in  the  US., 
Mexico,  Central  and  South  America,  and 
elsewhere  in  the  world.  Examining  the 
conditions  in  which  community  devel- 
opment and  education  occur  in  this 
country,  in  contrast  to  developing 
nations,  is  part  of  the  larger  work  of 


thinking  of  human  learning  and  devel- 
opment overall.  How  people  come 
together  and/or  change  lives  in  their 
communities  is  always  contextualized 
and  connected  to  particular  conditions. 

I wonder,  too,  how  much  the  agenda 
for  positive  social  change  is  generated  by 
literacy  workers,  and  how  much  com- 
munity development  work  hasn't  yet 
occurred  because  people  in  communities 
are  unaware  of  their  options.  How  do 
prevailing  conditions  (such  as  welfare 
regulations,  access  to  clear  and  readable 
information)  perpetuate  this  lack  of 
awareness?  In  whose  interest  is  informa- 
tion made  available  or  not?  Does  knowl- 
edge equal  power,  and  is  community 
development  solely  directed  at  access  to 
power?  Does  access  to  literacy  necessar- 
ily lead  to  greater  access  to  that  power? 
Power  for  whom?  Gained  how?  Hofer 
and  Larson  list  important  questions  to 
consider  in  finding  ways  to  assess 
"improvement  and  change  as  a continu- 
ous process",  for  individuals  [and]  the 
community  of  individuals";  in  addition, 

I still  want  to  know  why  we  assume  that 
adult  learners  might  be  interested  in 
community  and  leadership  develop- 
ment 

What  about  those  people  who  are 
afraid  to  challenge  the  status  quo  be- 
cause of  their  own  histories  as  immi- 
grants or  refugees?  What  about  those 
learners  who  seem  or  profess  to  be 
unaffected  by  conditions  which  others 
would  challenge?  Do  we  preach?  Do  we 
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politicize?  In  whose  interest? 

The  theme  of  community  itself  is  one 
which  learners  may  or  may  not  want  to 
discuss;  they  may  or  may  not  see  any 
importance  in  examining  their  notions  of 
community.  Writing,  drawing,  making 
maps  of  people's  routes  to  school, 
shopping  routes,  streets,  social/ visiting 
routes  or  talking  about  community  may 
lead  to  reflection  about  what  happens  in 
communities,  good  or  bad. 

As  adult  literacy  learners  become 
more  capable  readers  and  writers,  do 
they  then  become  more  capable  of  doing 
the  day  to  day  things  they  want  to  do? 
Do  other  possibilities  become  visible,  so 
these  learners  can  choose  to  become 
more  active  participants  in  the  lives  of 
their  communities?  Will  they  want  to 
take  on  issues  that  may  concern  them 
only  indirectly?  Is  it  our  goal  to  help 
people  see  the  links  between  seemingly 
unrelated  conditions  and  their  own 
lives? 

THI  RAINMAKER  PROJECT 

In  responding  to  Hofer  and  Larson, 

I want  to  briefly  describe  an  intergenera- 
tional  literacy  project  currently  entering 
its  second  year  in  Vancouver,  BC. 

The  Rainmaker  Project  was  envi- 
sioned by  Lee  Weinstein  as  a community 
development  effort  where  literacy 
would  be  a means  through  which 
intergenerational  learning  would  occur, 
where  people  in  an  inner-city  neighbor- 
hood would  come  together  to  address 
their  own  literacy  needs,  and  where 
adults'  and  children's  language  develop- 
ment would  be  supported.  The  program 
was  designed  to  serve  the  community 
surrounding  Macdonald  Elementary 
School  in  east  Vancouver,  as  well  as  to 
meet  the  needs  of  children  at  the  school. 


The  school  has  a population  of  approxi- 
mately 255  children,  of  whom  half  are 
First  Nations  (native  Indian),  40%  Asian 
(Chinese  and  Vietnamese),  5%  Spanish 
speaking  and  5%  Canadian-born  of 
Northern  European  origin.  The  school 
contains  grades  1 through  7,  as  well  as 
one  ESL  class.  School  staff  includes 
classroom  teachers,  first  nations  resource 
workers,  a family  advancement  worker, 
two  learning  assistance  center  teachers,  a 
project  teacher  and  a child  care  worker. 

A guiding  principle  in  the  project's 
relationship  to  the  community  is  the 
notion  of  breaking  down  isolation  and 
enabling  people  to  feel  connected  to 
their  world.  A parallel  objective  is  to 
develop  a center  and  educational  pro- 
cesses which  could  be  adapted  and 
eventually  owned  by  community  stake- 
holders. 

In  the  first  year,  project  staff  worked 
to  develop  opportunities  for 
intergenerational  learning  — for  parents, 
children  and  other  adults  in  the  schoo 
and  community.  Leamer-generatec; 
materials  were  a necessary  focus  ard 
product  of  our  work.  Computers  func- 
tioned as  a primary  vehicle  enabling 
staff  and  learners  to  create  necessary 
texts,  including  ongoing  documenta- 
tion/logs of  daily  classes  and  events, 
adults'  and  children's  stories  and 
classwork,  as  well  as  documents  created 
by  and  for  others  in  the  school  (newslet- 
ters, permission  forms,  announcements, 
grant  applications,  invitations). 

During  its  first  year,  over  240  children 
and  40  adults  worked  in  the  center,  a 
room  containing  two  round  tables, 
chairs,  book  shelves,  reading,  writing 
and  drawing  materials,  eight  Macintosh 
LC  computers,  and  two  printers.  (The 
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computers  and  one  of  the  printers  were 
obtained  through  a grant  from  Apple 
(Canada).  Children  in  the  school  (both 
mainstream  students  and  children  for 
whom  English  is  an  additional  language) 
worked  on  computers  in  the  project 
center  during  regularly  scheduled 
weekly  periods  (usually  45  minutes). 

The  center  was  open  to  classes  in  the 
school  five  days  a week,  and  maintained 
drop-in  hours  for  children  every  day 
after  school,  from  3 to  4 p.m.  In  addition, 
adults  (with  or  without  children)  had 
access  to  the  center  from  4-6  p.m.  Mon- 
days, Tuesdays,  Thursdays,  from  4-8 
p.m.  on  Wednesdays,  and  from  2 to  4 
p.m.  on  Fridays.  Children  came  to  the 
center  after  school  to  do  homework  as 
well  as  to  draw,  write,  play  games  and/ 
or  to  spend  time  with  peers  and  staff. 

We  utilized  the  computer  as  a tool 
which  can  help  children  and  adults 
express  ideas,  draw  pictures,  write, 
revise,  edit.  We  used  standard  word 
processing  and  drawing  programs  rather 
than  (with  few  exceptions)  educational 
software.  Children  began  by  using 
drawing  programs  to  acclimate  them- 
selves to  using  the  mouse  and  becoming 
comfortable  with  the  computers. 
Through  using  the  technology,  children 
particularly  in  ESL  and  kindergarten 
classes  — who  were  reticent  to  speak  in 
their  regular  classes  became  quite  en- 
gaged in  what  they  were  doing,  and 
spoke  to  others  to  share  their  findings, 
and/or  to  ask  how  to  do  more  or  differ- 
ent things. 

Parents  who  were  studying  English  as 
an  additional  language  in  the  room 
every  morning  were  quite  interested  in 
seeing  their  children  use  the  computers, 
although  most  preferred  to  stay  at  the 


conversation  table.  The  adults  who 
studied  in  the  morning  (from  9 to  1 1 :30) 
were  mothers  of  children  in  the  school, 
and  had  little  previous  involvement  with 
the  school  Most  of  these  mothers  would 
not  have  sought  out  more  'traditional' 
ESL  classes  at  other  sites  (community 
colleges,  adult  learning  centers)  but  were 
drawn  to  our  center  because  of  its 
location  at  their  children's  school.  As 
well,  child  care  was  provided  on-site  and 
there  were  no  fees  involved  for  either 
classes  or  child  minding. 

In  addition  to  parents  participating  in 
English  language  classes  during  the 
mornings,  other  Canadian-born,  English- 
speaking  parents  used  the  computers  on 
a drop-in  basis  to  write  letters,  poems, 
resumes,  and  songs,  and  also  to  do  work 
for  other  teachers  in  the  school.  As  the 
center  became  better  known  to  people  in 
the  community,  there  was  a steady 
increase  in  its  use,  and  adults  were 
increasingly  comfortable  coming  to 
either  watch  their  children  and/or  to 
work  alongside  them  learning  to  type, 
exploring  software  options,  writing 
themselves. 

During  its  first  year,  the  center  was  an 
active  place.  Part  of  our  purpose  is  to 
enable  community  people  to  increas- 
ingly define  the  goals  of  the  program. 
Ultimately,  our  mandate  is  to  have  the 
project,  should  it  continue  to  be  funded, 
run  entirely  by  people  of  the  community 
itself.  This  process  will  take  some  time 
and  involves  ongoing  support  from  the 
team  and  the  school.  As  of  this  writing,  it 
is  likely  that  some  form  of  the  project 
will  continue  within  the  school,  and 
other  funding  has  been  secured  to  start  a 
similar  project  within  another  nearby 
elementary  school 
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INDICATORS  OS  IFFICTTV1NESS 

A learning  center  which  allows  par- 
ticipants to  identify  their  own  needs,  to 
seek  assistance  when  and  as  needed,  and 
which  provides  both  staff  and  equip- 
ment as  resources  is  an  asset  to  a com- 
munity. When  that  learning  center  can 
open  its  doors  to  the  community  at  large, 
it  has  a real  opportunity  to  facilitate 
learning,  literacy  and  community  devel- 
opment. We  have  seen  the  increase  of 
participation  in  the  program,  both 
among  children  and  adults  at  the  center. 
We  now  need  to  demonstrate  the 
program's  effectiveness  and  to  evaluate 
this  progress  in  ways  that  meet  funders' 
requirements  while  also  remaining 
consistent  with  our  program  philosophy. 

In  terms  of  the  kinds  of  evaluative 
questions  literacy  workers  may  ask  are 
the  following:  Does  helping  Gwen  write 
her  many  letters  constitute  community 
development?  Does  Lawrence's  phone 
list?  What  about  the  Christmas  cards 
generated  by  him  and  his  family?  What 
about  kids'  exposure  to  the  possibilities 
of  technology?  We  seem  to  agree  that  we 
don't  want  tightly  sequenced  computer 
programs.  We  do  see  some  kids  writing 
for  their  own  authentic  purposes  — S's 
note  to  her  substitute  teacher,  A's  letters 
to  her  mom.  Many  kids,  though,  still 
resist  writing,  and  now  that  the  novelty 
of  the  technology  has  worn  off,  we  are 
faced  with  dilemmas  similar  to  those 
faced  by  classroom  teachers.  How  do  we 
argue  compellingly  for  kids  to  write?  Is 
it  through  reading  to  them?  Showing 
them  possibilities?  Asking  them  to  take 
their  work  seriously  and  to  read  to  one 
another?  Are  they  inspired  by  seeing 
adults  who  are  not  teachers  use  the 
technology  for  their  own  purposes?  In 
some  ways  we  are  very  limited  by  the 
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fact  that  we  do  not  work  in  classrooms, 
and  are  seen  as  a support  to  classroom 
activity  (and  in  some  cases  a diversion 
from  it).  As  we  continue  to  slowly  and 
carefully  build  connections  to  teachers 
and  students  at  the  school,  we  may  be 
able  to  bridge  some  of  that  distance.  We 
must  move  carefully  and  slowly;  this 
process  of  building  trust  is  essential  to 
the  work  we  hope  to  do. 

One  way  of  viewing  assessment, 
raditionally,  has  been  to  see  whether 
those  goals  have  been  met.  Who  sets 
these  goals?  for  whom?  who  measures 
growth?  how?  As  the  literature  on 
alternative  assessment  continues  to 
expand,  and  as  alternative  assessment 
itself  becomes  its  own  norm  among 
adult  literacy  practitioners,  new  ques- 
tions about  links  between  classroom 
work  and  community  action  are  being 
raised.  Does  participating  in  a literacy 
center  where  one  can  write  letters  to 
social  service  and  housing  agencies,  to 
newspaper  editors  and  policymakers 
constitute  a part  of  community  develop- 
ment per  se?  Does  the  awareness  that 
these  letters  can  be  written  constitute  a 
change  in  behavior?  Does  this  writing 
work  to  break  down  isolation  in  some 
tangible  way? 

A PERSONAL  VIEW 

I don't  live  in  the  community  in  which 
I work.  I'm  a middle  class  white  woman 
working  among  women,  men  and 
children  from  Canada,  Asia  and  South 
America  in  an  inner  city  neighborhood. 
How  do  I deal  with  the  conflict  I feel 
between  wanting  Asian  women  to  learn 
about  First  Nations'  concerns  and 
events?  Who  am  I to  say  that  helping 
Asian  and  Hist  Nations  parents  come 
together  is  a desirable  goal?  What 
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concerns  are  shared  by  those  ethnic 
communities  within  their  geographic 
community?  Can  collective  action 
around  such  concerns  as  safety,  security, 
heavy  traffic  on  streets  near  the  school, 
and  drug  use  bring  people  together? 
Does  problem  posing  work  within  the 
conversation  group  facilitate  the  devel- 
opment  of  collective  action?  Can  literacy 
work  around  neighborhood  flyers  and 
newspapers  help  to  increase  conscious- 
ness about  both  problems  and  opportu- 
nities for  change  within  the  community? 

Questions  about  learning  and  commu- 
nity development  seem  inevitably  linked 
to  questions  about  our  own  roles  as 
residents,  no-residents  of  continuities, 
towns,  cities,  countries.  Thinking  about 
returning  to  Vancouver,  I realize  again 
why  I went  there  to  work  two  years  ago. 
The  political  agenda  among  many  of  my 
colleagues  is  far  more  explicit  and 
articulated  than  was  my  own  two  years 
ago  when  I first  arrived  in  BC.  As  I 
struggle  to  name  important  processes, 
even  within  this  writing,  I realize  that  I 
need  to  learn  more.  Much  of  the  work 
done  by  adult  educators  in  the  US  and 
Canada  is  funded  by  sources  for  whom 
the  agenda  is  clearly  economic.  Adult 
learners  are  being  trained  into  jobs  they 
may  not  want  to  do,  or  worse,  still,  into 
jobs  which  may  not  exist  Practitioners  in 
either  country  must  be  able  to  articulate 
why  the  political  agendas  which  under- 


pin funding  may  be  racist  or  discrimina- 
tory and  to  help  other  adults  be  able  to 
articulate  these  problems,  too.  Many 
adults  in  communities  are  already  quite 
capable  of  naming  these  problems,  but 
may  feel  they  lack  access  to  those  people 
(policymakers,  politicians)  through 
whom  change  might  be  enacted. 

Programs  which  allow  adults  and 
children  to  work  together  seem  to 
facilitate  positive  growth  in  communi- 
ties. Programs  which  offer  the  potential 
for  community  people  to  run  the  centers 
in  which  they  began  as  learners  enable 
positive  growth  to  move  one  step  fur- 
ther. Finding  ways  to  articulate  what 
that  growth  is  and  how  it  helps  people 
and  communities  over  time  seems  the 
logical  next  step  in  the  course  of  plotting 
assessment  that  truly  tells  us  and  the 
learners  with  whom  we  work  where 
we're  going  — by  helping  people  look  at 
where  they've  been  and  at  where  they 
decide  they  need  to  go.  As  I struggle  to 
complete  even  this  essay,  I realize  I've 
raised  far  more  questions  than  I can  or 
should  hope  to  answer  at  this  moment. 

Author's  note:  The  Rainmaker  Project  team 
has  included  Lee  Weinstein,  Louie  Ettting, 
Rani  Gill,  Manabu  Seki,  Anne  McDonald, 
Dwayne  MacKenzie,  and  Nancy  Goldhar. 
Pieces  of  this  writing  have  been  adapted  by 
other  writing  Lee  Weinstein  and  I have  done 
about  the  project. 
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ESL  Assessment  Conundrum 


Bring  us  a test  that  is  normed. 

Not  to  mention  perfectly  formed. 

Fit  for  advanced  beginners. 

Lets  all  learners  be  winners. 

Spits  out  numerical  score 
That  provides  funders  with  lore. 

Hard  data  to  track. 

Offers  legislators  feedback. 

Standardization  might  do, 

Troyd  talk  to  Kalamazoo, 

Transition  starts  to  make  sense. 

And  teachers  wouldn't  feel  tense 

When  asked  to  document  progress 
For  more  accountability  not  less. 

They  know  students  are  advancing. 

But  have  trouble  substandng 

What  they  observe  everyday. 

From  "repeat"  to  role  play. 

Gains  in  confidence,  each  try. 

That's  good  enough!  Why  ask  why? 

Assessment  alternatives  ripe  to  explore 
Will  skills  transfer  beyond  the  classroom  door? 
Placement,  process,  progress  and  change. 

How  to  shape  our  stories  for  the  validity  range? 

— Diane  Pecoiaro 

Minnesota  Department  of  Education 
TESOL‘94 
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Book  Review 

Dimensions  of  Change 


Dimensions  of  Change: 

An  Authentic  Assessment  Guidebook 

Melody  Schneider  and  Mallory  Clarke 
1993  ABLE  (Adult  Basic  and  Literacy 
Educators);  Available  from:  ABLE,  1701 
Broadway,  Seattle,  Washington  (206)  587- 
3880 

The  new  ABLE  publication. 

Dimensions  of  Change:  An  Authen- 
tic Assessment  Guidebook , has 
much  to  recommend  it.  As  the  final 
documentation  of  a special  project 
entitled  'Integrated  Assessment:  Being 
Accountable  to  Teachers  and  Students," 
there's  an  integrity  and  authenticity  to 
this  resource.  Reading  Dimensions  of 
Change  made  me  nostalgic  for  the  days 
when  Massachusetts  had  discretionary 
federal  "353"  funds  that  went  out  to 
practitioners,  which  is  how  Melody 
Schneider  and  Mallory  Clarke  got 
money  to  do  such  in-depth  work.* 
Though  the  authors  never  specify  who 
they  mean  by  the  "diverse  adult  educa- 
tion programs"  — 14  teachers  and  70 
students  in  Washington  State  — they 
worked  with,  it  appears  that  the  guide- 
book is  intended  for  ABE,  not  ESL 
programs,  since  oral  language  assess- 
ment per  se  is  excluded  from  the  con- 
tents. However,  a great  deal  of  the 
book — from  theory  to  practice— am 
certainly  be  adapted  to  ESL  literacy. 

As  a "guidebook,"  the  text  fits  into  the 
genre  of  teacher  resources  I like  best  It 
combines  theory  and  practice;  it  pays 


attention  to  process  and  product;  it 
invites  flexible  use;  it  tells  stories  to 
illustrate  things;  it  has  lots  of  white 
space,  and  it  has  a spiral  binding  that 
allows  you  to  lay  it  flat  on  the  copy 
machine  to  reproduce  the  dearly-format- 
ted plethora  of  usable  "tools"  at  the  end. 
As  such.  Dimensions  of  Change  can  serve 
as  an  excellent  staff  development  guide 
for  programs  interested  in  re-examining 
their  current  assessment  practices  and 
trying  out  some  new  things.  I liked,  too, 
that  the  authors  chose  to  define  their 
focus  as  "authentic  assessment"  rather 
than  "alternative  assessment" , a term 
that  has  become  less  useful  as  "alterna- 
tive" approaches  enter  more  mainstream 
practices.  They  define  authentic  as 
"teachers  and  students  working  together 
to  find  multiple  ways  to  understand 
their  progress  and  problems  in  attaining 
educational  goals." 

Two  main  tenets  drive  the  work: 
partidpatory  education  and  whole 
language  practice.  The  authors  are  dear 
and  straightforward  in  the  presentation 
of  their  view  of  literacy;  this  is  not  an 
"anything  goes"  compilation  of  assess- 
ment protocols.  Chapter  1,  What  is 
authentic  assessment,  establishes  prin- 
dples  of  authentic  assessment  and 
elaborates  on  each  of  them.  Again,  there 
is  nothing  new  here,  but  it's  nicely 
presented,  and  consistent  with  current 
thinking  in  the  field  of  literacy  studies. 
They  remind  us  that  authentic  assess- 
ment is  learner  centered.  It's  an  inte- 
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grated  part  of  teaching  and  learning.  It 
reflects  the  complexity  of  learning;  it 
guides  teachers  in  daily  instruction.  It 
involves  real,  purposeful  activities.  And 
it  uses  a variety  of  methods.  It's  refresh- 
ing that  they  locate  their  focus  clearly  on 
teachers  and  learners  and  do  not  attempt 
to  muddy  the  process  by  dealing  with 
the  demands  of  funders  and  other 
stakeholders.  That  dialogue  is  a separate 
one,  with  different  issues  and  challenges. 
This  resource  focuses  on  the  teaching  / 
learning  process.(  I was  surprised, 
though,  that  Susan  Lytle's  work  wasn't 
cited  in  this  chapter,  since  she  was  an 
important  forerunner  of  much  of  this 
content,  and  influenced  many  people 
currently  doing  assessment  in  adult  ed. 

Chapter  1 reminds  me  again  of  the 
importance  of  starting  with  an  under- 
standing of  what  we  mean  by  literacy — 
and  by  extension,  how  we  should  be 
teaching  reading  and  writing — before 
we  jump  into  adopting  the  newest 
assessment  approaches.  Unless  teachers 
clarify  the  theoretical  base  of  whole 
language  and  the  political  and  practical 
considerations  of  participatory  educa- 
tion, the  niftiest  handouts  in  die  world 
aren't  going  to  be  very  useful 

The  authors  validate  teachers'  experi- 
ences by  reminding  them  that  "you 
already  do  authentic  assessment,"  a 
perspective  I have  been  sharing  with 
teachers  in  my  staff  development  work 
over  the  last  several  years.  They  note 
that  "authentic  assessment  provides  a 
way  for  teachers  who  are  assessing 
internally  to  document  their  observa- 
tions and  analyses,  making  them  us- 
able." This  is  a pivotal  issue  in  the  area 
of  assessment  It's  not  that  teachers 
aren't  doing  the  assessment,  it's  that  they 


are  not  documenting  it  in  ways  that  help 
them,  and  others,  see  how  students  are 
progressing.  Even  though  teachers  are 
the  best  suited  to  make  evaluations  of 
students'  progress,  their  analyses  are 
often  dismissed  by  funders  and  by 
teachers  themselves  as  "too  subjective," 
and  thus  not  valid.  By  analyzing  and 
documenting  their  observations  by  using 
a common  language  and  agreed  upon 
criteria,  teachers  can  indeed  present 
valid  assessments  of  students'  work  to 
themselves  (for  planning  and  instruct- 
ing), to  students  (to  see  their  own 
progress  and  set  goals)  and  to  other 
stakeholders  in  literacy  programs.  In  the 
long  run,  such  systematic  documenta- 
tion might  better  persuade  funders  to 
accept  approaches  like  portfolios  instead 
of  ineffectual  standardized  tests.  But 
that's  another  story... 

With  refreshing  candor,  the  authors 
share  their  own  "adventures  in  assess- 
ment" in  Chapter  2.  Here,  we  get  to  see 
some  of  the  stuff  that  didn't  work  and 
what  they  learned  from  it  This  realistic 
glimpse  into  process  is  another  example 
of  how  the  book  is  linked  to  real  class- 
rooms. 

If  programs  are  interested  in  going 
through  a process  of  their  own  in  order 
to  develop  better  assessment  practices. 
Chapter  3 will  be  especially  valuable. 
Specific  tasks  guide  the  reader  through  a 
process  in  developing  an  authentic 
assessment  system.  Two  things  stood 
out  for  me  in  this  section.  Hist,  the 
authors  realize  that  student  participation 
takes  some  time  to  cultivate,  encourage, 
and  promote.  Second,  they  recognize 
that  a project  like  this  demands  time  and 
money  and  suggest  ways  to  fund  it. 

One  program's  path  through  the 
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process  of  developing  authentic  assess- 
ment is  illustrated  in  Chapter  4.  This 
case  study  makes  things  really  concrete, 
and  includes  teachers'  journal  excerpts, 
transcriptions  of  discussions,  and  reflec- 
tion pieces.  Teachers  will  recognize 
questions  and  concerns  that  mirror  their 
own.  For  example,  a discussion  about 
the  use  of  checklists: 


Ellen:  I like  the  class  to  move  more 
organically  and  I'm  concerned  about 
the  time  it  takes  to  do  assessment 


m *-  . T*.  1 1 1 A. C11 t 
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the  lists  and  I'd  rather  use  the  time 
elsewhere.  I have  to  learn  how  to  get 
people  into  it 

Ellen:  We  haven't  set  up  things  to  make 
the  self  assessment  work.  We  have  to 
do  it  regularly.  Students  need  to  be 
able  to  be  really  reflective. 


Finally,  the  appendix.  This  satisfy- 
ingly  thick  section  of  tools  is  organized 
well  — first  by  categories  of  tools  and 
then  cross-referenced  by  placement  in 
the  class  cycle  (beginning,  on-going, 
final).  There  are  short  explanations  and 
commentaries  about  the  different  catego- 
ries of  tools,  too.  I've  seen  many  of  these 
protocols  in  similar  formats  floating 
around  in  a variety  of  sources,  but  it's 
nice  to  have  them  compiled  in  an  orga- 
nized format 


An  annotated  list  of  articles  and  books 
that  the  authors  used  when  researching 
their  project  is  located  at  the  end  of  the 
guidebook.  The  authors  claim:  "We  offer 
short  comments  from  our  personal 
perspective  as  readers  and  teachers.  We 
make  no  claims  that  you  will  agree  with 
us.  That's  the  way  it  goes  with  reading." 

Those  sympathetic  to  reader-response 
theory  (myself  included)  will  agree.  But  I 
was  bothered  by  the  unnecessarily  glib 
and  dismissive  manner  in  which  some  of 
the  resources  were  treated.  It  seemed 
uncharacteristic  of  the  rest  of  the  book 
that  extended  respect  to  teachers  strug- 
gling with  this  new  process  of  authentic 
assessment.  It  felt  like  I was  reading  over 
the  shoulder  of  the  authors  as  they  wrote 
to  each  other  here  without  a wider 
audience  considered.  Future  editions  of 
the  text  might  warrant  a rethinking  of 
this  section. 

* Note:  "353"  money,  federal  adult  education 
funding  earmarked  for  staff  development  and 
special  projects,  has  been  pooled  in  Massa- 
chusetts to  fund  the  SABES  system  as  a 
whole.  SABES  does,  of  course,  offer  mini- 
grants  for  special  projects.  Schneider  and 
Clarke  were  able  to  get  a grant  of$20j000  to 
fund  their  work.  I think  it's  important  to 
note  that  work  of  this  nature  does  require 
substantial  funding. 
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S A B E S 

System  tor  Adutt  Basic  Education  Support 


. SABES  is  the  System  for  Adult  Basic  Education  Support , a comprehensive 
training  and  technical  assistance  initiative  for  adult  literacy  educators  and 
programs.  Its  goal  is  to  strengthen  and  enhance  literacy  services  and  thus  to 
enable  adult  learners  to  attain  literacy  skills. 

SABES  accomplishes  this  goal  through  staff  and  program  development  work- 
shops, consultation,  mini-courses,  mentoring  and  peer  coaching,  and  other 
training  activities  provided  by  five  Regional  Support  Centers  located  at  commu- 
nity colleges  throughout  Massachusetts.  SABES  also  offers  a 15-hour  Orienta- 
tion that  introduces  new  staff  to  adult  education  theory  and  practice  and  enables 
them  to  build  support  networks. 

SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect,  evaluate, 
and  disseminate  ABE  materials,  curricula,  methodologies,  and  program  models, 
and  encourages  the  development  and  use  of  practitioner  and  learner-generated 
materials.  Each  of  the  five  SABES  Regional  Support  Centers  similarly  offers 
vrovram  surmnrt  and  a lending  library  SA.BES  maintains  an  Adult  Literacy 
Hotline,  a statewide  referral  service  whch  responds  to  calls  from  new  learners 
and  volunteers.  The  Hotline  number  is  1-800-447-8844. 


& 


The  SABES  Central  Resource  Center,  a program  of  World  Education,  publishes  a 
statewide  quarterly  newsletter,  "Bright  Ideas,"  and  journals  on  topics  of  interest 
to  adult  literacy  professionals,  such  as  this  volume  of  "Adventures  in  Assessment." 


The  first  three  volumes  of  "Adventures  in  Assessment"  present  a comprehensive 
view  of  the  state  of  practice  in  Massachusetts  through  articles  written  by  adult 
literacy  practitioners.  Volume  1,  Getting  Started,  includes  start-up  and  intake 
activities ; Volume  2,  Ongoing,  shares  tools  for  ongoing  assessment  as  part  of  the 
learning  process;  Volume  3,  Looking  Back,  Starting  Again,  focuses  on  tools 
and  procedures  used  at  the  end  of  a cycle  or  term,  including  self,  class,  and  group 
evaluation  by  both  teachers  and  learners.  Volume  4 covered  a range  of  interests, 
and  Volume  5,  The  Tale  of  the  Tools  is  dedicated  to  reflecting  on  Component  3 
tools  of  alternative  assessment.  Volume  6,  Responding  to  the  Dream  Confer- 
ence, is  dedicated  to  responses  to  Volumes  1-5.  Volume  7,  The  Partnership 
Project,  highlighted  writings  from  a mentoring  project  for  practioners  interested 
in  learning  about  participatory  assessment.  This  issue  covers  a range  of  topics 
from  education  reform  to  learner  involvement  in  assessment.  Volume  9 and 
future  issues  will  be  dedicated  to  specific  topics  ora  range  of  interests. 

Wed  like  to  see  your  contribution,  especially  if  you  are  doing  native  language  lit 
or  low-level  ESOL  lit.  Contact  Editor  Alison  Simmons  to  discuss  your  submission. 


Permission  is  granted  to  reproduce  portions  of  this  journal;  we  request,  however, 
appropriate  credit  be  given  to  the  author  and  to  Adventures  in  Assessment. 

Adventures  in  Assessment  is  free  to  DOE-funded  Massachusetts  programs; 
out-of-state  requests  will  be  charged  a nominal  fee.  Please  write  to,  or  call: 

Peppercorn  Books 
P.O.  Box  1766,  Durham,  NC  27702 
(919)  688-9313 
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Introduction 

Meeting  the  Challenge 
of  Authentic  Assessment 


by 

Alison  Simmons 


Editor 

SABES  Central 
Resource  Center/ 
World  Education 


One  of  the  greatest  challenges 
still  facing  adult  educators 
today  is  the  question  of  learner 
assessment.  How  will  I measure  the 
progress  of  my  learners  so  it  is  meaning- 
ful to  them,  informs  instruction  and  the 
curriculum,  and  satisfies  the  funders? 

In  the  past  seven  volumes  of  Adven- 
tures in  Assessment,  we  have  heard  from 
a variety  of  practitioners  sharing  their 
ideas,  successes  and  attempts  at  explor- 
ing and  developing  learner-centered 
approaches  to  assessment  and  evalua- 
tion. As  contributors  struggled  with  the 
question  of  what  "a  learner-centered 
approach  to  assessment  and  evaluation" 
meant,  they  offered  us  a different  way  to 
assess  and  test  our  learners,  and  helped 
us  think  beyond  the  confines  of  stan- 
dardized, commercial  instruments. 

In  the  age  of  education  reform,  un- 
stable funding,  measurable  outcomes, 
and  program  accountability,  we  who 
believe  in  authentic  assessment  feel  a 
need  to  defend  our  ways  of  measuring 
learner  progress  and  program  effective- 
ness to  the  powers  that  make  decisions 
about  the  future  of  the  field.  We  also 
want  to  "lift  the  burden  of  accountabil- 
ity off  the  backs  of  learners  and  place  it 
on  programs  and  funders"  (Whiton, 
Bright  Ideas,  Vol.  5,  #1,  Summer  '95). 
Within  these  challenges  we  hope  to  find 
ways  to  showcase  the  work  that  has 
already  been  done  in  authentic/ alterna- 
tive assessment  and  not  return  to  the  age 
of  assessment  that  gave  us  the  numbers 
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and  positive  outcomes  but  told  us  little 
of  program  effectiveness  or  the  personal 
gains  our  learners  made  as  parents, 
citizens,  and  workers. 

In  Volume  8 of  Adventures  in  Assess- 
ment (AiA),  Johan  Uvin  revisits  his 
earlier  article  (AiA,  Volume  1)  and  talks 
about  the  role  of  authentic  assessment  in 
workplace  education  programs  in  light 
of  education  reform.  Uvin  states  "Edu- 
cation reform  is  here  to  stay,  at  least  for 
the  foreseeable  future,  and  will  have  an 
impact  on  workplace  education,  includ- 
ing but  not  limited  to  learner  assessment." 

While  Uvin  gives  us  the  education 
reform  perspective,  Lisa  Levinson  talks 
about  a statewide  perspective  on  sys- 
temic change.  In  1991,  Maine  decided 
that  portfolio  assessment  was  to  replace 
standardized  testing  in  Adult  Education 
Programs.  (Sandy  Brawders  introduced 
us  to  Maine's  portfolio  in  AiA,  Vol  6.) 

Levinson,  from  the  Horizon  Project 
in  Maine,  takes  us  on  a chronological 
journey  through  Maine's  process  of 
developing  and  implementing  their 
State  Indicators  of  Program  Quality.  She 
looks  back  over  the  past  three  years  and 
documents  the  events  that  have  led  to 
the  implementation  and  development  of 
the  Indicators  and  portfolio  assessment 
as  an  integral  part  of  Maine's  adult 
education  system. 

In  the  next  article,  Yvonne  La  Lyre 
looks  at  the  broad  question  of  Native 
Language  Literacy  (NLL)  instruction 
and  assessment  through  a review  of  the 
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literature.  As  Native  Language  Literacy 
increasingly  becomes  a part  of  the  ABE 
system,  it  impacts  how  and  why  we 
assess  people  for  literacy  in  their  native 
language.  At  the  end  of  her  article, 
Yvonne  looks  at  how  programs  around 
the  country  are  assessing  their  NLL 
learners  and  what  resources  are  avail- 
able for  assessment  and  instruction. 

She  challenges  us  think  about  how  these 
findings  impact  our  ESOL  literacy  classes 
and  our  need  to  assess  learner  progress. 

Two  authors  address  the  question 
often  raised  by  adult  educators:  "How 
can  we  get  learners  more  involved  in 
the  assessment  process?"  Molly  Paul 
Nguyen  and  Michele  Vemi  explore 
different  ways  to  look  at  involving 
learners  in  the  dialogue  about  what 
assessment  is  and  what  the  purposes 
should  be. 

Nguyen  took  the  question  of  assess- 
ment and  testing  into  the  classroom  as 
a lesson  plan  to  start  a discussion  about 
assessment  in  the  lives  of  her  learners. 
This  discussion  prompted  her  to  alter 
her  assessment  practices  and  to  start 
thinking  about  other  ways  to  involve 
learners  in  the  assessment  process. 

Michele  Vemi  was  involved  in  a 
statewide  survey  of  adult  learners  that 
asked  how  adult  education  programs 
affected  their  lives.  She  talks  about  her 
experience  being  part  of  that  process 
and  leads  us  to  question  what  an  "adult 
learner"  perspective  is. 

Two  teachers  from  very  different 
programs  tackle  the  issue  of  creating  a 
common  assessment  tool  to  measure 
learner  progress  for  teachers  working 
for  the  same  program  but  at  different 
sites  or  different  levels.  Jenny  Utech 
talks  about  the  challenge  teachers  in  her 


program  had  coming  up  with  a tool  that 
could  satisfy  both  ABE  and  ESOL 
classes.  Jenny  raises  the  question  of 
whether  there  is  a pre-  and  post-assess- 
ment tool  or  process  that  could  satisfy 
the  needs  of  both  ABE  and  ESOL  learn- 
ers and  teachers  in  the  same  program 
at  the  same  time  with  not  only  varying 
goals,  but  also  varying  needs  in  terms 
of  language  acquisition  and  learning. 

Linda  Gosselin  et  al  tackled  the  same 
issue  in  their  program.  The  Quinsiga- 
mond  program's  need  to  streamline  its 
assessment  was  motivated  by  transfer- 
ring students  from  one  class  to  another. 
There  was  no  systematic  way  of  doing 
this  previously,  so  assessment  on  an 
ongoing  basis  and  at  the  end  of  the  term 
was  not  meaningful.  She  and  her  col- 
leagues were  looking  for  a tool  flexible 
enough  to  meet  the  individual  needs 
of  the  teachers  but  common  enough  to 
meet  program  and  funding  requirements. 

Two  authors  take  us  into  their  pro- 
grams and  give  us  a glimpse  of  how 
authentic  assessment  is  implemented. 
Caroline  Gear  discusses  the  need  for 
teachers  as  well  as  learners  to  have 
developmental  time  for  integrating 
authentic  assessment  in  their  classes 
and  what  support  teachers  need  to 
make  this  happen.  We  have  all  experi- 
enced the  initial  reluctance  of  learners 
who  are  new  to  authentic  assessment. 

It  takes  time  for  learners  and  teachers  to 
find  ways  to  make  the  process  participa- 
tory and  meaningful.  The  same  time  is 
needed  for  programs  to  work  with  staff 
who  are  new  to  authentic  assessment. 
Caroline  looks  at  one  authentic  assess- 
ment tool  and  talks  about  how  this  can 
be  a beginning  framework  for  teachers 
and  programs  new  to  authentic  assessment 
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Marti  Duncan  takes  us  quite  literally 
on  a tour  of  her  adult  education  center 
in  rural  Maine  where  she  shows  us 
portfolio  assessment  is  alive  and  well. 
Through  several  interviews  we  see  that 
portfolio  assessment  is  systemwide  and 
can  be  used  to  measure  more  that  just 
learner  progress.  We  should  look  at  it 
as  a process  that  everyone  in  a program 
can  participate  in  for  their  own  assess- 
ment purposes. 

In  What  Counts ? Paula  Carranza  et  al 
write  about  math  activities  that  can  be 
used  as  an  assessment  tool.  The  authors 
walk  us  through  a few  activities  that 
they  have  found  useful. 

We  reviewed  two  books  in  this  vol- 
ume. Sylvie  O'Donell  looked  at  Assess- 
ing Success  in  Family  Literacy  Programs,  a 
book  that  looks  at  designing  and  imple- 
menting family  literacy  programs. 


Eileen  Barry  reviewed  Whole  Language 
for  Adults:  Guide  to  Instruction,  Portfolio 
Assessment,  Initial  Assessment  and  Admin- 
istration and  Staff  Development.  This  is  a 
comprehensive  resource  for  programs 
and  has  in-depth  sections  on  initial 
assessment  tools  and  a guide  for  the  use 
of  portfolio  assessment  in  adult  educa- 
tion programs. 

There  are  a lot  of  ideas  here  — both 
broad  and  narrow  — that  will  hopefully 
spark  interest  in  others  to  write  about 
their  authentic  assessment  ideas  and 
tools.  AiA  has  always  been  one  of  the 
places  where  the  dialogue  of  authentic/ 
alternative  assessment  takes  place.  We 
invite  you  within  Massachusetts  and 
afar  to  share  your  ideas,  successes,  and 
questions  as  we  learn  from  each  other 
about  authentic  assessment.. 

S'  QAa^/1 
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In  light  of  education  reform 

Revisiting  Assessment  in 
Workplace  Education 


In  Volume  One  of  Adventures  in 
Assessment,  I wrote  about  evalua- 
tion for  workplace  education  pro- 
grams. At  that  time  I was  working  as  a 
Workplace  ESOL  (English  to  Speakers 
of  Other  Languages)  teacher  in  the 
workplace  education  program  at  the 
South  Cove  Manor  Nursing  Home  in 
Boston  s Chinatown.  The  article  docu- 
mented my  efforts  to  demonstrate  the 
effectiveness  of  the  program  by  develop- 
ing an  evaluation  plan  that  involved  all 
stakeholders  actively,  with  learner 
assessment  as  an  integral  but  challeng- 
ing part  of  that  work. 

The  1991  article  identified  the  de- 
mands for  accountability  articulated  by 
supervisors,  managers,  and  funders  as 
one  of  many  challenges.  I maintained 
that  a collaborative  approach  to  plan- 
ning and  evaluation  has  the  potential  to 
strike  a balance  between  what  learners 
and  teachers  demand  as  evidence  of 
progress  and  the  demands  expressed 
by  individuals  who  are  not  directly 
engaged  in  the  teaching  and  learning 
process. 

In  this  article  I revisit  the  assessment 
of  learners  in  workplace  education.  Two 
developments  make  revisiting  the  issue 
timely:  recent  changes  in  the  national 
policy  landscape  and  efforts  to  systemi- 
cally  reform  education  at  the  state  level. 
The  key  question  I am  concerned  with 
in  this  article  relates  to  the  impact  these 
developments  may  have  on  how  work- 
place education  teachers  approach  the 


assessment  of  learners.  Most  of  the 
article  is  devoted  to  discussing  the 
challenges  these  developments  create 
for  teachers. 

At  first  sight  these  challenges  appear 
to  add  more  pressures  to  teachers.  I 
conclude,  however,  that  the  opportuni- 
ties education  reform  efforts  provide  can 
enable  teachers  to  1)  further  strengthen 
the  use  of  program-based  authentic 
assessment  where  assessment  is  closely 
tied  to  the  classroom  instruction  and 
2)  separate  authentic  program-based 
assessment  from  statewide  assessment 
for  accountability  purposes. 

Workplace  Education  Is  Changing 

In  all  aspects  of  society,  the  demand 
for  accountability  is  increasing.  "It  is 
time  to  deliver,"  has  become  a standard 
phrase  in  legislative  debates  and  the 
media.  Positive  outcomes  and  delivera- 
bles are  the  key  factors  in  deciding 
whether  services  will  continue  to 
receive  support.  Government  is  reduced. 
Health,  education  and  human  services 
programs  are  rescinded,  if  not  elimi- 
nated, in  an  apparently  unstoppable 
urge  to  make  the  nation's  ends  meet. 
Reform  is  introduced  to  radically  change 
systems,  consolidate  programs  and 
reinvent  government  to  increase  effec- 
tiveness of  the  services  left.  Few  stones 
remain  unturned  and  workplace  educa- 
tion is  not  one  of  them. 

Workplace  education  is  changing. 
Governmental  support  (e.g.  in  the  form 
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A long-term  plan 
for  assessment 
in  adult  basic 
education  will 
be  developed 
to  assess  the 
feasibility  of 
the  10  th  grade 
assessment  and  to 
explore  program- 
based  portfolio 
assessment. 


of  categorical  funds)  for  workplace 
education  is  under  threat.  Rescissions 
of  workplace  education  funds  will 
result  in  the  elimination  of  the  largest 
(federal)  program  that  supports  work- 
place education  in  Massachusetts,  the 
National  Workplace  Literacy  Program, 
in  November  1997. 

A similar  scenario  is  unfolding  for 
discretionary  resources  for  workplace 
education  currently  available  under  the 
Job  Training  Partnership  Act  (8%  JTPA). 
The  Massachusetts  Labor  Shortage 
Initiative  that  supported  workplace 
education  and  training  of  staff  in  acute 
care  hospitals  has  been  phased  out. 

Block  grant  legislation  is  expected  to 
consolidate  some  of  these  discretionary 
funding  sources  but  does  not  earmark 
portions  of  block  grants  for  workplace 
education.  As  a result,  workplace  educa- 
tion must  make  a strong  case  to  get 
some  of  these  block  grant  funds.  The 
private  sector  — both  business  and  labor 
— will  experience  pressure  to  step  up 
to  the  plate  to  increase  its  investment 
in  workplace  education.  Assuming  the 
private  sector  will  somewhat  expand  its 
responsibility  for  supporting  programs, 
demands  for  accountability  may  shift. 
Teachers  may  find  themselves  increas- 
ingly working  for  businesses  instead 
of  education  agencies  and  will  witness 
first-hand  how  old  public  funder  de- 
mands for  accountability  are  disappear- 
ing and  are  replaced  by  demands 
defined  by  private  funders  who  to 
varying  degrees  may  value  other  learn- 
ing gains  than  those  connected  directly 
and  exclusively  to  the  productivity  of 
the  organization. 

National  and  state  education  reform 
efforts  (e.g.  Goals  2000:  Educate 
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America  Act  and  the  Massachusetts 
Education  Reform  Act  of  1993)  bring 
about  an  additional  wave  of  systemic 
change  that  will  touch  learners  and 
teachers  in  workplace  education  pro- 
grams. High  expectations  for  students, 
teachers,  counselors,  administrators, 
and  schools  (including  adult  learning 
centers)  are  introduced  to  ensure  that 
all  residents  of  this  nation  are  well- 
prepared  for  their  present  and  the 
future,  and  increasingly  demanding 
roles  as  parents,  workers  and  citizens. 

The  Massachusetts  Common  Core 
of  Learning  captures  the  essence  of  the 
educational  foundation  children  and 
adults  are  expected  to  have  acquired 
at  the  end  of  high  school  or  its  adult 
secondary  education  equivalent. 
Curriculum  frameworks  have  been 
developed  for  K-12,  and  the  develop- 
ment of  supplemental  adult  basic  educa- 
tion curriculum  frameworks  has  begun. 

. Based  on  the  content  standards 
embedded  in  the  frameworks,  a two- 
pronged assessment  system  is  being 
developed.  These  prongs  consist  of  a 
statewide  standardized  but  "perfor- 
mance-based" assessment  at  the  4th, 

8th  and  10th  grade  levels  for  statewide 
accountability  purposes,  and  a local 
school-based  component  of  portfolio 
assessment.  (A  long-term  plan  for 
assessment  in  adult  basic  education  will 
be  developed  to  assess  the  feasibility  of 
the  10th  grade  assessment  and  to  ex- 
plore program-based  portfolio  assess- 
ment.) 

The  Adult  Education  Committee 
established  by  the  Education  Reform  Act 
of  1993  (Sections  29  and  75)  has  defined 
the  level  of  need/ demand  for  and  sup- 
ply of  adult  basic  education  services  in 
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the  Commonwealth.  It  has  prepared 
recommendations  for  the  administration 
and  the  Legislature  for  the  key  elements 
and  funding  levels  of  a coordinated 
adult  basic  education  service  delivery 
system.  Industry  skills  standards  are 
being  developed  and  used  in  the  imple- 
mentation of  School  to  Work  initiatives 
and  in  local  workplace  education  cur- 
riculum development  efforts. 

This  list  reflects  only  a part  of  the 
reform  initiatives  that  are  underway 
but  demonstrates  clearly  that  education 
reform  is  comprehensive.  For  workplace 
education  teachers,  education  reform 
begs  the  questions:  What  does  it  all 
mean  ? Should  we  pay  attention  to  these 
additional  directions  we  feel  pulled  into 
or  should  we  continue  doing  what  we 
have  been  doing  for  years  and  hope 
education  reform  will  go  away  ? Is  there 
a role  for  us  in  "making  this  change" 
and  bringing  it  to  life  in  our  classrooms, 
in  the  way  we  teach  and  assess  learners? 

Challenges  and  Opportunities 

Education  reform  is  here  to  stay  at 
least  for  the  foreseeable  future,  and  will 
have  an  impact  on  workplace  education 
including,  but  not  limited  to,  learner 
assessment.  Education  reform  for  adult 
basic  education  including  workplace 
education,  however,  will  take  time  and 
its  ultimate  success  will  depend  on  the 
outcomes  of  the  changes  that  teachers 
have  brought  about  in  their  classrooms. 
Their  ability  to  reconcile  the  skills, 
needs,  interests  and  goals  of  adult 
learners,  the  needs  of  their  employers 
and  unions  and  the  high  expectations 
established  in  the  Common  Core  of 
Learning  and  fleshed  out  in  the  curricu- 
lum frameworks  will  be  critical. 
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Workplace  education  teachers  will 
not  have  to  do  away  with  the  way  they 
teach,  develop  curricula  or  assess  learn- 
ers. On  the  contrary,  their  teaching 
methodology,  their  success  stories,  their 
skills  in  developing  highly-customized 
and  contextualized  curricula  and  materi- 
als, and  their  ability  to  design  and 
implement  performance-based  assess- 
ments provide  an  outstanding  starting 
point  for  reviewing  the  adult  basic 
education  supplemental  curriculum 
frameworks  from  the  perspective  of  the 
adult  worker  and  his/her  employer  and 
union.  Once  developed,  the  frameworks 
will  provide  all  who  have  an  interest  in 
providing  workplace  education  services 

— both  in  the  private  and  public  sectors 

— with  content  standards  and  guide- 
lines for  developing  local  curricula  that 
use  the  actual  contexts  of  the  workplace 
and  the  experiences  of  workers  to  teach 
the  skills  implied  in  the  high  expecta- 
tions of  the  Common  Core. 

Assessment  of  learners  in  workplace 
education  has  always  been  challenging. 
An  important  reason  is  that  no  stan- 
dardized tests  are  available  that  mea- 
sure the  learning  gains  at  which 
workplace  education  programs  aim. 

This  has  challenged  and  enabled  work- 
place education  programs  to  demon- 
strate accountability  towards  their  goals 
that  goes  beyond  the  standardized  tests. 
Current  standardized  tests  fail  to  cap- 
ture the  specific  goals  of  programs  and 
instruction  which  in  most  instances  are 
driven  by  the  needs  of  the  local  work- 
place (i.e.  employer  and  employee). 
Standardized  assessments  also  assume 
that  there  is  an  agreement  on  content 
standards  across  programs.  To  date, 
that  consensus  has  been  lacking. 
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As  a result,  workplace  education 
teachers  have  developed  alternative 
ways  of  assessing  the  impact  of  instruc- 
tion on  student  learning.  While  reliabil- 
ity and  validity  have  been  concerns  in 
these  efforts,  teachers  have  been  able 
in  a considerable  number  of  cases  to 
generate  reliable  information  on  the 
impact  of  instruction  using  perfor- 
mance-driven instruments  where  learn- 
ers in  real  work  situations  demonstrate 
gains  by  applying  what  they  have 
learned.  The  use  of  classroom-based  job 
simulations,  or  the  analyses  of  video 
recordings  of  learners  at  different  points 
in  time  using  English  in  work-related 
interactions  are  just  two  examples  of 
authentic  assessment  instruments 
teachers  have  developed  and  used 
successfully. 

Education  reform  does  not  take 
away  the  challenge  of  developing 
appropriate  assessment  instruments  but 
provides  promising  ways  of  looking  at 
assessment  from  both  a program  and 
system  accountability  point  of  view. 
Once  adult  basic  education  supplemen- 
tal frameworks  are  available,  there  will 
be  agreement  on  content  standards  that 
then  gets  reflected  in  local  or  industry- 
based  curricula,  instruction  and  assess- 
ment. Assuming  that  the  10th  grade 
statewide  standardized  assessment  is 
designed  so  that  it  is  developmentally 
and  culturally-appropriate  for  use  with 
adult  learners,  this  performance-based 
assessment  could  then  be  used  for 
statewide  accountability  purposes. 

The  content  standards  embedded  in 
the  adult  basic  education  supplemental 
curriculum  frameworks,  however,  may 
not  all  be  addressed  in  local  or  industry- 
based  curricula.  Neither  must  they  be. 
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The  program  goals  and  design  (e.g. 
duration  of  instruction)  will  define 
which  standards  are  addressed.  The  use 
of  a program-based  portfolio  assessment 
approach  creates  opportunities  for 
learners  to  self-select  writings,  readings, 
project  reports,  video  and  audio  record- 
ings, courses  taken,  test  results  and 
other  assessments  taken  for  inclusion 
in  their  portfolio.  Reviews  of  portfolios 
by  learners  and  teachers  at  several 
points  in  time  would  then  reveal  which 
specific  learning  goals  were  achieved 
through  instruction  and  how  students 
are  progressing  over  time  towards 
acquiring  the  high  expectations  the 
Commonwealth  has  for  all  students. 

Conclusion 

Revisiting  assessment  in  workplace 
education  in  light  of  education  reform 
has  enabled  me  to  articulate  a possible 
but  broadly-defined  way  that  workplace 
education  teachers  can  connect  mean- 
ingfully with  and  substantially  inform 
education  reform  through  participation 
in  the  development  of  adult  basic  educa- 
tion supplemental  frameworks.  These 
frameworks  can  then  be  used  by  work- 
place education  teachers  to  develop  local 
or  industry-specific  curricula  and  pro- 
gram-based authentic  assessments  based 
on  the  specific  needs  of  adult  workers 
and  their  workplaces. 

During  the  Winter  of  1995  and  the 
Spring  of  1996, 16  study  groups  in 
Massachusetts  will  begin  developing 
supplemental  adult  basic  education 
frameworks.  Between  January  and  June 
of  1996,  additional  specialty  groups  will 
provide  input  from  the  perspective  of 
the  population  they  are  serving.  One  of 
these  specialty  groups  will  be  for  work- 
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place  education.  It  will  minimally 
consist  of  teachers  currently  involved  in 
the  curriculum  working  group  of  the 
Massachusetts  Workplace  Literacy 
Consortium.  They  will  review  curricu- 
lum frameworks  from  the  perspective  of 


the  adult  worker. 

For  further  information,  please  contact 
Adult  and  Community  Learning  Services  at 
the  Department  of  Education  at  (617)  388- 
3300  at  extension  349. 
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Maine's  History 
of  Systemic  Change 


by 

Lisa  Levinson 


Horizon  Project 
Center  for  Adult 
Learning  and  Literacy 
Orono,  ME 


The  beginning: 

In  1991,  Congress  passed  the 
National  Literacy  Act.  Although 
there  was  notice  of  its  enactment 
at  the  time,  there  was  not  much  that 
changed  in  Maine's  Adult  Education 
system.  It  is  only  now,  looking  back, 
that  we  realize  the  National  Literacy  Act 
was  a mandate  for  change  in  our  state. 

Two  of  the  provisions  contained  in 
the  National  Literacy  Act  were  the  for- 
mation of  a gubernatorially-appointed 
state  literacy  resource  center  and  a field- 
generated plan  of  Indicators  of  Program 
Quality.  State  literacy  resource  centers 
would  be  funded  by  the  Act,  and  states 
were  given  until  June,  1993  to  produce 
Indicators  of  Program  Quality  (what  we 
call  our  Quality  Indicators). 

In  November,  1992,  Sandy  Brawders 
became  a staff  development  specialist  at 
the  Staff  Development  Project  housed  at 
the  University  of  Maine.  She  had  been 
working  in  Massachusetts  as  a consult- 
ant to  that  state's  Department  of  Educa- 
tion and  had  been  responsible  for  its 
Quality  Indicator  process.  She  asked 
about  our  Quality  Indicators,  and  where 
we  were  in  developing  them.  Sandy  was 
then  asked  to  spearhead  Maine's  Quality 
Indicator  process.  The  rest,  as  they  say, 
is  history! 

Sandy  Brawders  trained  eight  field 
representatives  as  facilitators  for  small 
groups  and  invited  teachers,  tutors, 
directors,  counselors  as  well  as  other 
agency  representatives  to  attend  a 
March  30  Quality  Indicator  meeting. 
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Needless  to  say,  March  in  Maine  is  not 
the  best  time  to  be  assured  of  good 
weather.  In  spite  of  a raging  snow /ice 
storm,  75  people  from  all  over  the  state 
attended  our  first  Quality  Indicator 
meeting.  At  this  momentous  meeting,  it 
was  decided  that  Maine  would  abolish 
grade-level  labeling,  use  portfolio 
assessment  to  replace  the  reliance  on 
standardized  testing,  and  utilize  three 
levels  of  learning  for  placing  students. 
The  three  levels  were  based  on  the 
intensity  of  service  an  adult  learner 
would  need  to  progress  to  the  next  level. 
Intensity  Level  1 learners  needed  the 
most  intensive  service  because  they 
had  the  most  to  learn  and  accomplish. 
Intensity  Level  3 learners  needed  less 
intensive  service  because  they  had  more 
independent  learning  skills.  Fourteen 
Indicators  of  Program  Quality  were 
generated  at  this  meeting. 

A few  weeks  later,  a work  group 
that  included  the  meeting  facilitators 
met  in  a hotel  room  of  the  Howard 
Johnson's  in  Waterville  to  synthesize  the 
data  from  the  March  30th  meeting  into  a 
coherent  document.  It  became  clear  that 
there  were  consistent  themes  intersect- 
ing all  14  of  the  Indicators  that  had  been 
generated.  The  work  group  collapsed 
and  nested  the  14  Indicators  into  four, 
but  retained  the  information  of  all  14. 
The  resulting  Indicators  are:  Indicator  1, 
Learner  Expectations;  Indicator  2, 
Program  Planning;  Indicator  3,  Field 
Organization  Responsibilities;  Indica- 
tor 4,  State  Responsibilities. 


604 


Volume  8:  December  1995 


The  draft  document  with  the  four 
Indicators  went  out  to  the  field  for 
comment  through  17  public  hearings 
across  the  state  that  were  attended  by 
adult  learners,  teachers,  tutors,  adminis- 
trators and  other  agency  partners.  The 
draft  was  revised,  sent  out  for  comment 
again,  and  submitted  to  the  United 
States  Department  of  Education  on  June 
30, 1993.  Maine's  Quality  Indicator 
document  was  the  only  document 
approved  by  the  USDOE  without  revi- 
sion, and  has  served  as  a model  of 
inclusionary  visioning  for  other  states. 

Our  Quality  Indicator  document 
formed  two  important  groups.  The 
Council  for  Systems  Development 
(CSD),  an  interagency  group  that  was 
designed  to  oversee  the  implementation 
of  the  Quality  Indicators  across  agen- 
cies. It  has  been  meeting  since  June,  1993 
on  a bi-monthly  basis.  The  CSD  has 
solidified  the  connections  between  Adult 
Education,  ASPIRE  and  the  Bureau  of 
Employment  and  Training.  This  group 
has  coordinated  joint  ventures  of  col- 
laboration and  continues  to  do  so. 

The  ABE  Liaison  Committee  was 
also  born  of  the  Quality  Indicators.  Its 
function  is  to  help  the  Maine  Division  of 
Adult  and  Community  Education  revise 
the  Request  for  Proposals  (RFP)  process 
for  Adult  Basic  Education  to  reflect  the 
Quality  Indicators.  This  committee 
helped  revise  the  RFP  process  in  1994, 
and  completely  rewrote  the  RFP  in  1995. 

Implementation: 

In  Summer,  1993  the  Adult  Educa- 
tion Academy  was  held  at  Sunday  River 
in  Newry,  Maine.  Traditionally  an  Adult 
Education  directors  meeting,  it  was 
opened  up  to  include  teachers  and 
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tutors.  At  the  Summer  Academy,  the 
field  generated  the  ethical  framework 
we  all  would  use  in  order  to  begin 
developing  contextualized  portfolio 
assessment  throughout  our  state.  The 
Golden  Rules  of  portfolio  enabled  us  to 
begin  the  inquiry  process  into  portfolio 
assessment  that  was  grounded  in  the 
same  understanding.  The  first  rule  of 
our  Golden  Rules  is  that  the  learner  owns 
their  portfolio.  This  has  become  a very 
important  tenet  in  our  state. 

The  1993  Summer  Academy  was 
also  the  site  of  the  initial  planning  of  the 
New  England  Literacy  Resource  Center 
(NELRC).  The  New  England  states  had 
decided  that  it  would  be  in  the  best 
interests  of  all  six  states  to  pool  some 
of  each  state's  literacy  resource  center 
money  (based  on  a proportional  for- 
mula) to  create  the  NELRC.  This  agree- 
ment by  all  six  states'  governors  was 
approved  by  the  United  States  Depart- 
ment of  Education  in  Spring,  1993. 

It  was  decided  that  all  six  states  would 
get  together  to  work  on  developing  state 
needs  assessments  in  October. 

At  the  October  meeting,  Maine 
traveled  to  Sturbridge,  MA  with  a van 
load  of  ten  Adult  Educators  to  develop 
a framework  for  conducting  the  needs 
assessment  for  a Resource  Center  in 
each  state.  Maine  chose  to  hold  regional 
focus  groups  as  well  as  to  devise,  dis- 
seminate and  collect  a survey.  The 
regional  focus  groups  were  comprised  of 
learners,  practitioners,  tutors,  adminis- 
trators and  other  agency  caseworkers, 
trainers  and  teachers.  The  survey  went 
to  teachers,  directors  and  learners  across 
the  state.  The  resulting  document.  Yes, 
We  Can!  articulated  the  vision  of  what 
adult  education  should  be  in  Maine. 


The  first  rule  of 
our  Golden  Rules 
is  that  the  learner 
owns  their 
portfolio.  This 
has  become  a very 
important  tenet 
in  our  state. 
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Our  Quality  Indicator  document  has 
many  x's  where  resources  and  training 
were  necessary  to  create  Indicators  of 
Quality.  Additional  resources  were 
needed  to  be  able  to  fill  in  the  x's.  The 
Staff  Development  Project  was  deluged 
with  requests  for  portfolio  development 
workshops.  Sandy  Brawders  produced 
the  Why  Portfolio  document  that  in- 
cluded the  "Golden  Rules,"  resources, 
and  suggested  starting  points.  In  a 
concerted  effort  to  keep  up  with  the 
demand,  Sandy  Brawders  trained  ten 
members  of  the  field  to  help.  Out  of  that 
training,  another  foundation  document 
was  created:  The  Educational  Autobiogra- 
phy. This  document  showed  the  connec- 
tion of  portfolio  to  all  four  Indicators, 
and  how  the  learner  was  the  center  of 
it  all. 

The  Horizon  Project 

In  August,  1993  our  state  literacy 
resource  center  (the  staff  development 
project)  applied  for,  and  in  October 
received,  a two-year  discretionary  grant 
from  the  National  Institute  for  Literacy 
to  build  statewide  capacity  for  an  inter- 
agency staff  development  system.  This 
was  another  attempt  to  provide  re- 
sources to  fill  in  those  x's.  The  Horizon 
Project  became  a reality. 

The  Horizon  Project  piloted  a re- 
gional approach  to  staff  development  by 
creating  seven  regions  in  the  state.  Each 
region  had  a trainer,  or  Horizon  Cata- 
lyst, who  provided  support  for  regional 
alliance  development  and  staff  develop- 
ment opportunities  around  the  four 
areas  of  the  grant:  portfolio  assessment, 
evaluation,  reporting  and  teacher  re- 
search or  inquiry.  These  four  areas  were 
the  key  elements  that  intersected  all  four 
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Quality  Indicators.  The  Horizon  Project 
was  an  inquiry  project  itself,  with  the 
Catalysts  developing  their  skills  with  the 
field.  Their  motto  was  to  be  the  'guide  at 
the  side,'  not  the  'sage  on  stage.'  Portfo- 
lio assessment  became  a major  focus  of 
the  grant,  and  the  regional  model  helped 
build  interagency  connections  and 
helped  start  and  sustain  the  portfolio 
inquiry  process. 

Other  Initiatives 

At  the  same  time,  the  state  received  a 
three-year  correctional  education  grant 
to  develop  lifeskills  curricula  at  seven 
county  jails  in  Maine.  The  Esteem  Ma- 
chine would  develop  a life  skills  curricu- 
lum that  modeled  integrating  academic 
and  life  skills  using  portfolio  assess- 
ment. The  curriculum  modules  would 
be  based  on  the  most  common  reasons 
for  incarceration.  The  Esteem  Machine 
has  been  developing  curriculum  with 
inmates  using  an  inquiry  method  ever 
since.  They  have  taken  not  only  correc- 
tional education  to  new  heights  but 
have  shown  the  rest  of  the  field  what 
can  happen  when  learners  are  partners 
in  curriculum  development.  This  grant 
also  fills  in  some  of  the  X's  in  the  Qual- 
ity Indicator  document. 

The  State  Literacy  Resource  Center 
became  known  as  the  Center  for  Adult 
Learning  and  Literacy  (CALL)  in  Febru- 
ary of  1994.  CALL  began  publishing  a 
monthly  newspaper  called  the  Adult 
Education  Newsletter  that  quickly 
became  the  main  connection  to  what 
was  happening  in  adult  education  on 
the  national,  state  and  local  levels.  It 
contained  articles  from  the  field,  from 
learners  and  from  other  agencies.  After  a 
'name  the  newspaper'  contest,  and  as  an 
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April  Fool's  joke,  the  Maine  Fertilizer 
debuted  in  April.  Its  name  caused  quite 
a controversy,  but  no  one  forgot  it!  It 
continues  to  be  a forum  for  discussion, 
a showcase  for  local  programs  and  a 
resource  for  national  news  and  pro- 
grams. Special  inserts  have  highlighted 
other  agencies.  Horizon,  Esteem  Ma- 
chine and  NELRC  work.  The  Fertilizer 
currently  has  a circulation  of  over  6,000 
readers.  For  the  first  time,  teachers, 
learners  as  well  as  directors  in  the  adult 
education  system  and  other  agencies 
were  receiving  consistent  news  about 
aciviif  gducstion. 

Next  Steps 

The  Quality  Indicator  document 
required  that  the  field  meet  every  year 
to  assess  the  process  of  implementing 
the  Indicators  and  to  plan  for  the  next 
steps.  In  March,  1994,  the  Next  Step 
meeting  produced  a document  that 
outlined  what  still  needed  to  be  done  to 
help  the  field  realize  the  Indicators.  At 
this  meeting,  learners  were  included  for 
the  first  time  and  began  to  form  a state- 
wide learner  group.  This  group  began 
producing  the  Maine  Idea  , a newspaper 
of  learner  writings  and  letters  that  was 
distributed  as  an  insert  in  the  Fertilizer. 

The  Summer  Academy  of  1994 
became  the  Summer  Institute.  CALL  and 
Horizon  staff  created  a "Den  of  Inquiry" 
that  modeled  using  methods  that  were 
inclusive  of  multiple  intelligences. 
Teachers  highlighted  their  "Go  Getter" 
projects  at  a curriculum  fair.  The  "Go 
Getter"  projects  were  teacher  projects 
funded  by  CALL  to  develop  curriculum 
that  modeled  using  portfolio  as  the 
center  of  their  teaching.  The  Maine  Math 
Motivators,  our  state's  team  that  was 


examining  the  Math  Standards  and 
alternative  ways  to  teach  math,  also 
provided  demonstrations  and  projects. 
Learners  attended  the  Institute  for  the 
first  time,  and  produced  the  first  copy  of 
the  Maine  Idea  here.  The  first  interagency 
strand  brought  together  adult  education 
programs  and  ASPIRE  (JOBS)  regional 
managers  to  talk  about  developing 
courses  that  met  the  needs  of  ASPIRE 
clients.  Several  Memorandum  of  Under- 
standing contracts  were  developed  as  a 
result  of  that  meeting,  and  interagency 
referrals  increased  by  25%. 

The  Summer  Institute  sparked  a lot 
of  creativity  in  the  field  over  the  next 
year.  Portfolio  assessment  was  becom- 
ing a reality  in  a lot  of  programs.  Learn- 
ers were  leading  the  way  by  presenting 
their  portfolios  to  teachers  and  talking 
about  how  beneficial  the  process  was  to 
their  learning  and  self-esteem.  The  field 
was  writing  up  their  experiences  for  the 
Fertilizer.  Horizon  was  making  connec- 
tions in  the  field  and  providing  training. 
CALL  was  providing  training  and  con- 
nections for  new  teachers  and  directors, 
and  developed  tool  kits  to  help  collect 
resources  and  materials  on  subjects 
pertinent  to  program  success.  The  Divi- 
sion of  Adult  and  Community  Educa- 
tion was  supporting  the  changes  and 
highlighted  them  at  state-wide  meetings. 
It  seemed  that  every  time  you  turned 
around  something  was  happening. 

The  March,  1995  Last  Step  to  Imple- 
mentation Quality  Indicator  meeting 
once  again  brought  the  field  together  for 
change.  The  Last  Step  toward  imple- 
mentation was  to  develop  standards  for 
our  four  Indicators.  The  first  standards 
to  be  developed  would  be  for  Indicator 
1:  Learner  Expectations.  At  this  meeting. 
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Learners  were 
leading  the  way 
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their  portfolios 
to  teachers  and 
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the  process  was 
to  their  learning 
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Maine  adopted  the  four  principles 
outlined  in  the  preliminary  paper  by 
Sondra  Stein  of  the  National  Institute 
for  Literacy  as  the  four  standards  for  our 
state  for  Quality  Indicator  1:  Literacy 
as  Access  and  Orientation;  Literacy  as 
Voice;  Literacy  as  Independent  Action; 
and  Literacy  as  Bridge  to  the  Future. 
That  day,  learners,  teachers,  tutors, 
directors  struggled  with  creating  bench- 
marks for  the  four  standards. 

The  staff  of  CALL  and  Horizon 
synthesized  the  field  input  from  that 
day  into  a draft  standards  document. 

The  document  included  all  of  the  field 
generated  data  from  the  March  meeting 
including  creating  five  levels  of  learning 
and  teaching,  not  just  3.  It  was  clear 
from  the  field  input  that  there  needed  to 
be  a pre-literacy  level  as  Level  1 and  a 
post  secondary  level  as  Level  5 to  reflect 
all  of  the  populations  Adult  Education 
has  as  customers.  In  June,  there  were 
five  public  hearings  around  the  state 
to  solicit  feedback  on  the  draft. 

The  1995  Summer  Institute,  "Chart- 
ing Our  Course,"  revolved  around  the 
standards  document  draft.  Learners, 
teachers,  tutors,  counselors,  administra- 
tors, Department  of  Labor  JTPA  provid- 
ers and  state  officials  worked  together 
to  examine  the  standards  draft,  provide 
input  for  another  revision  and  examined 
interagency  ties.  As  a result  of  the 


Institute,  Maine  is  in  the  process  of 
developing  one  set  of  standards  for 
adult  education,  ASPRE  (JOBS)  and  job 
training  including  JTPA.  We  are  doing 
this  together  with  all  the  other  agencies 
that  work  with  educating  and  training 
adults. 

So,  how  much  change  has  there  been 
in  the  last  three  years?  We  have  changed 
our  method  of  assessment,  refused  to 
grade-level  adults,  developed  new 
teaching  methodologies  using  portfolios 
as  the  center,  developed  life  skills 
curriculum  in  partnership  with  students, 
instituted  a continuous  grassroots 
process  for  change,  developed  several 
state-wide  teams  and  many  teacher 
projects  using  an  inquiry  model,  institu- 
tionalized a state-wide  adult  education 
newspaper,  developed  a student  leader- 
ship group  called  the  Life  Long  Learn- 
ers, delivered  staff  development  on  an 
on-going  basis  instead  of  the  "one  shot" 
model,  created  regional  alliances  that 
include  other  agencies,  produced  tool 
kits  that  are  accessible  to  anyone, 
changed  the  RFP  for  Adult  Basic  Educa- 
tion to  reflect  the  Quality  Indicators, 
created  and  sustained  a key  interagency 
group  called  the  CSD  and  adopted  four 
Quality  Indicators  and  four  standards 
for  Indicator  1. 

Just  think  of  what  the  next  three 
years  will  bring! 
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Issues  in  Assessment  of 
Native  Language  Literacy 


Native  Language  Literacy 

(NLL)  instruction  for  adults  is 
increasingly  becoming  part  of 
Adult  Basic  Education  (ABE).  A small 
but  growing  number  of  programs 
offering  English  for  Speakers  of  Other 
Languages  (ESOL)  also  offers  adults 
basic  literacy  instruction  in  their  mother 
tongue.  In  1992  3 national  survey  uncov- 
ered  68  such  programs  in  the  continental 
U.S.,  an  increase  of  400%  during  the 
previous  eleven  years. 

The  main  reason  for  the  increase 
seems  to  be  that  ESOL  teachers  and 
program  directors  are  recognizing  that 
there  needs  to  be  a program  or  class  that 
addresses  the  population  that  does  not 
respond  to  instruction  at  the  most  basic 
level  of  English.  Parallel  to  that,  the 
current  literature  in  second  language 
acquisition  and  bilingual  education 
unequivocally  supports  the  thesis  that 
having  reading  skills  in  the  native 
language  (LI)  facilitates  reading  com- 
prehension in  the  second  language  (L2). 
Most  recently  it  has  been  suggested  that 
the  acquisition  of  L2  oral  skills  may  also 
be  influenced  by  reading  skills  in  LI 
(Earl,  1994).  There  are  also  reports  that 
learners  in  NLL  programs  become  more 
involved  in  the  lives  of  their  community 
and  children  (Gillespie,  1994). 

Despite  the  promise  of  the  NLL 
approach,  the  controversy  surrounding 
bilingual  education  and  consequent  lack 
of  funding  have  contributed  to  the  lack 
of  research  in  this  new  field.  Most  NLL 


programs  have  developed  as  grassroots 
responses  to  the  needs  of  ethnic  commu- 
nities and  have  struggled  with  the 
development  of  teaching  strategies, 
assessment,  placement  and  curriculum, 
with  no  resources  left  for  research.  As 
more  NLL  programs  are  established, 
the  interest  and  need  to  learn  from  them 
increases.  The  National  Center  on  Adult 
Literacy  and  the  Center  for  Applied 
Linguistics,  responding  to  that  need, 
have  published  several  documents 
describing  NLL  programs  across  the 
country.  Here  I will  draw  from  such 
documents  and  my  survey  of  programs 
in  Massachusetts  to  provide  the  reader 
with  a discussion  and  summary  of  some 
of  the  issues  posed  by  the  assessment 
and  instruction  of  adult  learners  of 
Native  Language  Literacy.  (This  discus- 
sion does  not,  however,  include  Native 
American  Adult  Literacy.) 

Who  qualifies  as  an  NLL  learner? 

Candidates  for  NLL  instruction  are 
adults  who  speak  a language  other  than 
English  as  their  mother  tongue  and  have  . 
attended  0-5  years  of  elementary  school. 
Although  they  may  have  acquired 
"survival  English  skills",  they  are  over- 
whelmed by  the  difficulties  or  memories 
of  negative  experiences  presented  by  a 
school-like  environment  and  teaching 
which  relies  on  the  use  of  the  black- 
board, books,  and  grammatical  explana- 
tions. Many  deny  their  condition  to 
avoid  the  stigma  associated  with  illit- 
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If  the  minimal 
goal  of  an  ESOL/ 
ABE  program  is 
to  provide  adults 
with  the  necessary 
skills  to  " lead  a 
meaningful  life 
in  the  community 
where  they  live  in 
the  USA, " then 
the  acquisition 
of  reading  and 
writing  skills 
in  English  are 
essential. 


eracy  and  rely  on  individually-created 
devices  to  cope  with  the  demands  of 
their  community  life.  Even  if  their 
strategies  successfully  allow  them  to 
participate  in  the  life  of  a community 
where  perhaps  the  majority  speaks  their 
native  language,  they  are  effectively 
barred  from  accessing  the  services 
offered  by  most  ESOL/ABE  programs 
(Klassen,  1991, 1993).  They  are  also 
not  apt  to  enroll  in  programs  that 
offer  instruction  to  obtain  the  GED  in 
a language  other  than  English,  such  as 
Spanish  and  French  simply  because  the 
level  of  instruction  is  too  demanding.  It 
is  conceivable  that  many  of  these  adults, 
aware  of  the  unavailability  of  services, 
refrain  from  contacting  ESOL  programs. 

Within  the  ESOL  program  the  NLL 
learner  is  among  the  students  who  make 
little  or  no  progress  at  the  lower  levels. 
As  other  immigrants  who  have  deferred 
receiving  ESOL  instruction,  s/he  may 
have  developed  some  sort  of  "pidgin" 
English  that  allows  him/her  to  commu- 
nicate at  work,  at  stores,  and  hospitals, 
but  when  it  comes  to  understanding  and 
acquiring  the  patterns  of  standard 
English  used  in  a classroom,  s/he  faces 
insurmountable  difficulties. 


The  Big  Picture 

If  the  minimal  goal  of  an  ESOL/ABE 
program  is  to  provide  adults  with  the 
necessary  skills  to  "lead  a meaningful 
life  in  the  community  where  they  live  in 
the  USA,",  then  the  acquisition  of  read- 
ing and  writing  skills  in  English  are 
essential.  By  far  the  hardest  hurdle  to 
cross  in  this  process  is  to  ascertain  what 
needs  to  be  done  and  how  much  can  be 
accomplished  with  the  resources  avail- 
able. Assessment  not  only  of  individual 
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learner  skills  and  goals  but  also  assess- 
ment of  the  community  and  the  collec- 
tive group  of  learners  entering  your 
program  needs  to  happen  first  to  see  if, 
in  fact,  an  NLL  class  is  feasible. 

At  least  two  situations /signs  can 
indicate  the  need  for  native  language 
literacy:  1)  a large  group  of  immigrant 
adults  with  low  literacy  skills  and  from 
different  language  backgrounds,  or 
2)  a large  group  of  adults  lacking  lit- 
eracy skills,  from  similar  cultural  back- 
grounds. Since  most  programs  have 
limited  capacity  to  address  all  the  needs 
of  all  learners,  the  preferred  strategy  to 
address  the  first  situation  is  to  create  an 
ESOL  literacy  class,  where  whole  lan- 
guage and  participatory  methodology 
will  be  used.  The  staff  of  an  ESOL/ABE 
program  may  also  opt  for  providing 
some  sort  of  bilingual  instruction  to 
those  adults  with  the  aid  of  tutors,  or  for 
referring  learners  to  a program  that  does. 

To  address  the  second  situation, 
approaches  to  NLL,  as  described  in  the 
literature  (Gillespie,  1994,  Wrigley,  et  al, 
1992)  adhere  to  the  following  patterns: 

1.  Sequential  (or  transitional)  in  which 
learners  receive  native  language 
literacy  until  they  reach  a certain 
level  of  literacy  and  then  transfer  to 
an  ESOL  beginners  class. 

2.  "Bilingual",  in  which  both  the  native 
language  and  English  are  used  in  the 
instruction  of  literacy  in  English.  In 
this  case  the  native  language  is  used 
to  translate  the  meaning  of  words 
and  to  explain  aspects  of  grammar. 

3.  Coordinate,  in  which  learners  re- 
ceive NLL  instruction  and  primarily 
oral  English  skills  training  in  sepa- 
rate classes,  taught  by  two  different 
teachers. 
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Most  NLL  instruction  is  a combina- 
tion of  the  "bilingual"  and  the  coordi- 
nate approaches,  since  the  adults  rarely 
want  to  wait  to  begin  receiving  instruc- 
tion in  English.  Most  existing  NLL 
programs  are  for  speakers  of  Spanish, 
although  there  are  programs  in  other 
languages  such  as  Haitian  creole,  Hmong, 
and  Khmer. 

With  all  NLL  approaches,  several 
issues  with  respect  to  resources,  learners 
and  outcomes  arise.  Lack  of  materials  in 
the  language  of  the  learners  is  the  most 
often  articulated  problem.  Among 
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books  used  in  Latin  American  countries 
and  in  bilingual  programs  in  the  U.S. 
with  the  constraint  that  most  are  inad- 
equate for  two  reasons.  One  is  that 
many  adults  lacking  literacy  skills  speak 
a regional  Spanish  — whose  vocabulary 
may  vary  considerably  from  one  region 
of  Latin  America  to  another  — and  not 
what  is  disseminated  in  textbooks.  The 
second  reason  is  that  the  materials  are 
geared  for  the  instruction  of  children 
and  therefore  ignore  the  rich  cognitive 
background  that  adults  bring  to  the 
acquisition  of  reading  skills. 

Among  Hmong  programs  materials 
are  mainly  teacher-made  although  there 
are  resources  available  from  California, 
Minnesota,  and  Wisconsin  where  large 
groups  of  Hmongs  and  Laotians  have 
settled.  In  Haitian  creole  there  is  also 
a lack  of  standard  materials  since  it  is 
fairly  recent  that  the  language  has  been 
coded  in  its  written  form.  These  circum- 
stances have  had  the  positive  outcome 
of  forcing  teachers  and  programs  to 
develop  curricula  that  respond  to  the 
linguistic  and  cultural  needs  of  the  adult 
learners  in  the  particular  setting  of  an 
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immigrant  in  the  U.S.  In  Milwaukee,  for 
example,  where  there  is  a large  concen- 
tration of  Hmong  speakers,  a 0-4  partici- 
patory curriculum  was  developed 
through  a collaboration  between  the 
Milwaukee  Area  Technical  College 
and  the  Lao  Family  Community,  Inc. 

In  Massachusetts,  Spanish  native 
language  literacy  programs  offer  either 
the  sequential  or  the  coordinate  ap- 
proach, depending  on  the  goals  of  the 
program.  The  Puerto  Rican  Cultural 
Center  in  Springfield,  for  example, 
offers  the  sequential  model  and  Alianza 
Hispana  in  Boston  offers  the  coordinate 
model.  Other  programs  combine  ap- 
proaches. At  the  Haitian  Multi-Service 
Center  (HMSC),  for  example,  students  in 
level-one  receive  literacy  instruction  in 
Haitian  creole  at  the  same  time  that  they 
receive  mathematics  in  a combination  of 
English  and  Haitian  creole.  In  the  level- 
two  class,  literacy  and  mathematics  are 
taught  in  two  languages. 

At  the  Community  Learning  Center 
(CLC),  students  begin  in  a class  where 
reading  is  taught  in  both  Haitian  creole 
and  English.  That  is,  the  teacher  uses 
vocabulary  from  both  languages  to 
show  the  relationship  between  the 
spoken  and  the  written  word.  The 
proportion  of  Haitian  creole  and  English 
used  in  the  classroom  shifts  from  75%/ 
25%  to  25%/75%  by  the  end  of  the  term. 
At  that  time  the  students  shift  to  a begin- 
ner's English  class  with  the  option  to 
continue  the  bilingual  literacy  class  until 
they  feel  comfortable  taking  only  English. 

At  the  two-level  Hmong-Lao  Foun- 
dation Literacy  Program  in  Fitchburg, 
adults  learn  to  read  and  write  in  Hmong 
while  receiving  extensive  pronunciation 
skills  training  in  English,  since  speaking 
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As  can  be  ex- 
pected, since  the 
goal  of  most  NLL 
programs  is  to 
assist  learners  to 
acquire  English 
skills,  standard- 
ized tests  in 
English  are  the 
most  common 
instruments  used 
to  gauge  progress 
at  the  end  of  any 
term  or  cycle. 


English  is  what  poses  the  most  difficul- 
ties for  Hmong  adults. 

Implications  for  assessment 

Assessment  is  tightly  linked  to  the 
goals  and  methodology  of  teaching  in  a 
program.  As  can  be  expected,  since  the 
goal  of  most  NLL  programs  is  to  assist 
learners  to  acquire  English  skills,  stan- 
dardized tests  in  English  are  the  most 
common  instruments  used  to  gauge 
progress  at  the  end  of  any  term  or  cycle. 
Methods  and  tools  used  by  programs  to 
assess  initial  and  ongoing  development 
of  verbal  skills  vary  widely,  however, 
adjusted  to  goals  and  resources  of  each 
one.  McGrail  (1992),  in  a summary  of 
assessment  in  native  language  literacy 
programs  in  the  U.S.,  mentions  that  the 
most  developed  form  of  assessment  is 
the  initial  intake.  I have  divided  the 
following  discussion  into  two  sections: 
Spanish  language  programs  and  others, 
since  the  majority  of  NLL  programs  are 
in  Spanish. 


Initial  Assessment:  Spanish  programs 
Among  Spanish  programs  the 
California  Test  of  Basic  Skills  (CTBS)  in 
Spanish  and  the  Spanish  Assessment  of 
Basic  Education  (SABE)  are  the  most 
popular  since  they  provide  a grade  level 
equivalent  score.  However,  they  are 
designed  for  children,  not  for  adults. 

The  single  test  designed  for  adults  is 
the  ABLE  in  Spanish.  Recently  a Spanish 
translation  of  the  TABE  has  been  pub- 
lished but  has  no  record  of  performance 
in  the  literature.  Another  drawback  of 
standardized  tests  is  that  the  learners 
unfamiliar  with  multiple-choice  formats 
may  be  assessed  unfairly.  As  a result  of 
the  lack  of  adequate  standardized  tests. 
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programs  use  a combination  of  stan- 
dardized tests  and  teacher-made 
instruments  to  obtain  information  for 
placement  decisions.  It  is  especially 
common  to  all  programs  to  have  an 
informal  method  to  assess  writing  skills, 
mainly  to  be  sensitive  to  the  feelings  of 
inadequacy  that  low-literacy  adults  have. 

The  intensity  and  scope  of  the 
assessment  can  vary  significantly  ac- 
cording to  the  resources  of  the  program. 
Not  all  NLL  programs  can  afford  to 
offer  a continuum  of  levels  or  support 
that  allows  learners  to  transfer  ad- 
equately from  the  native  language  to  an 
ESOL  class  and  the  initial  assessment  is 
reflective  of  that  aspect  of  the  program. 

At  a small  program  in  Massachu- 
setts, for  example,  the  teacher  asks 
adults  to  read  aloud  from  a Spanish 
beginning  reader  series,  to  write  a few 
phrases  and  solve  a few  simple  arith- 
metic problems.  Subsequently  she  places 
the  adult  in  one  of  two  groups  compris- 
ing the  entire  program. 

In  contrast,  at  the  ten-year  old  El 
Paso  Community  College  Literacy 
Education  Action  Program,  staff  mem- 
bers observe  verbal  language  domi- 
nance, writing  skills,  and  ability  to 
complete  forms,  at  an  initial  screening 
through  which  they  determine  whether 
the  adult  has  a reading  level  above  the 
sixth  grade  in  his/her  native  language 
(Gillespie  & Ballering,  1992).  Anyone  at 
that  level  is  referred  to  an  ESOL  pro- 
gram while  others  begin  a two-week 
exploration  class  in  which  the  learner's 
goals  and  acquired  skills  are  carefully 
examined  before  placement.  A student 
in  that  program  may  eventually  choose 
to  follow  a vocational  track  or  prepare 
for  college. 
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Programs  that  offer  NLL  in  Spanish 
up  to  the  level  of  GED  preparation  and 
ESOL  may  test  a student  in  both  lan- 
guages. If  a student  arrives  requesting 
GED  preparation,  the  initial  assessment 
is  in  Spanish  and  the  student  is  placed 
in  the  appropriate  group  level  and  may 
continue  that  track  up  to  the  ASE  level. 
Staff  will  use  either  a standardized  test, 
or  a process  or  instrument  created  at  the 
program  to  determine  when  the  learner 
has  achieved  a sixth-grade  level  of 
reading  comprehension.  (The  Boston 
Public  Schools  battery  of  cloze  tests  is 
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for  that  purpose.)  At  that  level  the 
learner  is  deemed  able  to  enroll  in  a 
beginning  ESOL  class.  If  a learner 
requests  ESOL  at  intake,  his/her  Span- 
ish literacy  level  is  assessed  along  with 
English  listening  and  reading  compre- 
hension and  written  and  oral  proficiency. 


Other  Programs 

The  Haitian  Multi-Service  Center 
(HMSC)  program  in  Boston,  widely 
described  in  the  literature,  has  histori- 
cally responded  to  the  needs  of  its 
learners.  An  assessment  instrument 
including  an  oral  interview,  reading 
comprehension  and  writing  (developed 
by  Loren  McGrail  and  Marie  Julien)  that 
includes  English  and  French  serves  as 
the  preliminary  placement  test.  Those 
whose  educational  background  and 
performance  in  this  initial  interview 
demonstrate  the  ability  to  progress  in 
ESOL  classes,  are  placed  at  the  appropri- 
ate level.  At  a second  stage  of  assess- 
ment the  Creole  Biliteracy  teacher,  Louis 
Daniel,  interviews  the  learner.  During 
this  interview,  Daniel  indirectly  obtains 
the  information  necessary  to  determine 

ERK — — 

QQS&SI olunie  8:  December  1995 


whether  or  not  the  person  can  read  or 
write  and  where  to  place  him/her  in 
the  two-level  biliteracy  program. 

Similarly,  at  the  CLC  in  Massachu- 
setts and  the  Hmong-Lao  Foundation  in 
Fitchburg,  the  learner  skills  in  both  the 
native  language  and  English  are  exam- 
ined by  a bilingual  staff  member.  At  the 
Lao  Family  Community  in  Milwaukee, 
an  initial  brief  interview  to  determine 
the  student's  educational  background  is 
followed  by  the  pro-gram's  placement 
test.  In  addition,  the  BEST  for  English 
skills  and  the  bilingual  TABE  for  math- 
siristics  °ivsn  to  satisfy  tho  rpnnirp- 
ments  of  funding  sources. 

Besides  asking  adults  the  level  of 
schooling  they  have  attained  at  intake,  a 
program  may  choose  to  use  the  Native 
Language  Literacy  Screening  Device 
developed  by  Dr.  Patricia  Mooney- 
Gonzalez  in  New  York,  available  in 
several  languages  (Arabic,  Chinese, 
English,  French,  Haitian  Creole,  Italian, 
Polish,  Portuguese,  Russian,  Spanish, 
Tigrinya,  and  Vietnamese)*.  Feedback 
has  been  scarce  but  positive  from  ESOL 
programs  in  Massachusetts  which  have 
used  it.  One  ESOL  program  director 
reported  that  it  helped  her  to  place 
students  and  eventually  convinced  her 
of  the  need  to  offer  NLL  instruction. 


* The  instrument,  along  with  instructions, 
can  be  obtained  by  contacting  any  of  the 
SABES  Regional  Centers.  The  reader  may 
also  contact  the  A.L.R.I./SABES  Boston 
Regional  Center  for  additional  literature  and 
assessment  instruments  which  have  been 
compiled  by  Maria  E.  Gonzalez,  617-782- 
8956. 
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On-going  assessment 

Spanish  programs  that  offer  only  a 
few  levels  of  NLL  do  not  use  standard- 
ized testing  to  transfer  a student  from 
one  level  to  another.  In  small  programs 
and  those  which  concentrate  on  devel- 
oping reading  and  arithmetic  skills  in 
the  native  language,  on-going  assess- 
ment emphasizes  teacher  observations 
and  some  sort  of  portfolio  assessment 
that  includes  writing  skills. 

Sequential  programs  that  offer  NLL 
up  to  GED  preparation  in  Spanish  have 
some  form  of  "in  house"  assessment 
process  to  determine  when  students 
have  a competence  level  in  the  native 
language  that  will  allow  them  to 
progress  in  the  ESOL  classes;  they  may 
also  use  standardized  tests.  Essentially 
the  student  may  not  take  ESOL  classes 
until  the  sixth  grade  reading  compre- 
hension level  in  the  native  language  is 
achieved.  In  some  cases,  the  initial 
assessment  tool  is  used  throughout  the 
program  to  gauge  the  progress  of  the 
student.  At  the  Casa  Aztlan  program  in 
Chicago,  for  example,  a staff-developed 
test  examines  six  levels  of  basic  skills 
and  is  used  at  the  end  of  each  of  three 
eight-week  modules.  At  the  Puerto 
Rican  Cultural  Center  and  many  others, 
the  SABE  in  Spanish  serves  as  the 
standardized  measure  which,  in  con- 
junction with  teacher  observations  and 
portfolio  assessment,  is  used  to  recom- 
mend program  changes  twice  a year. 

In  coordinate  models,  where  ESOL 
instruction  is  provided  early  in  the 
program,  ongoing  assessment  in  NLL 
has  less  relevance  for  ESOL  program- 
matic changes.  For  example,  at  Alianza 
Hispana  in  Boston,  NLL  students  at  the 
lowest  level  are  allowed  to  enroll  in  the 
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beginning  ESOL  class  after  the  first  ten- 
week  term  is  over.  At  Oficina  Hispana, 
those  students  are  enrolled  immediately 
in  the  beginning  ESOL  class  but  are 
required  only  to  "audit"  the  class.  As 
teachers  observe  their  progress  in  the 
NLL  classes,  they  are  exhorted  to  be- 
come increasingly  active  participants 
in  the  ESOL  classes. 

More  than  one  teacher  has  com- 
mented to  me,  that  a student  able  to 
read  at  the  third  grade  level  in  the  native 
language  is  able  to  read  at  that  same 
level  in  English  but  lacks  the  grammati- 
cal competence  to  comprehend  beyond 
that  level.  As  a result,  teachers  of  the 
ESOL  beginning  classes  know  not  to 
expect  rapid  progress  among  students 
with  basic  skills  below  a sixth  grade 
level. 

As  in  small  Spanish  programs, 
ongoing  assessment  for  NLL  students 
is  not  standardized  in  programs  offered 
in  Haitian  creole  and  other  languages  in 
which  standardized  testing  in  the  native 
language  is  irrelevant.  In  these  pro- 
grams crucial  assessments  are  those 
that  can  help  determine  when  a com- 
plete transition  from  NLL  to  English  or 
to  ESOL  instruction  is  reasonable.  The 
degree  of  testing  structure  seems  to 
vary  according  to  the  scope  of  the 
program  as  well.  At  the  HMSC  and  the 
CLC,  which  are  large  programs  with  a 
history  of  stability  and  where  students 
may  continue  in  the  NLL  class  practi- 
cally as  long  as  they  wish,  teacher-made 
tests  and  observations  and  portfolio 
assessment  are  used  to  coordinate  NLL, 
basic  skills,  and  ESOL  instruction.  At  the 
Lao  Family  Community,  where  learners 
receive  literacy  and  basic  skills  instruc- 
tion in  Hmong,  ESOL,  and  GED  prepa- 
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ration,  and  the  curriculum  is  compe- 
tency-based, teachers  use  pre-and  post 
tests  in  Hmong  produced  at  the  institu- 
tion and  tailored  to  the  population  it 
serves. 

Final  assessment 

Final  assessment  in  the  native  lan- 
guage may  be  the  sixth-grade  reading 
level  measure  that  indicates  the  ability 
to  transfer  and  begin  an  ESOL  class  or 
the  GED  examination  in  those  languages 
where  it  is  made  available.  When  the 
program  is  modeled  after  the  transi- 
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reading  level  is  not  the  last  hurdle  for 
the  students.  At  Mujeres  Unidas  en 
Accion  in  Boston,  the  GED  in  Spanish  is 
the  final  step;  at  the  Puerto  Rican  Cul- 
tural Center,  the  sixth  grade  reading 
level  marks  the  beginning  of  ESOL  and 
the  GED  the  final  grade.  At  the  PRCC 
and  at  Alianza  Hispana,  adults  may 
continue  the  ESOL  sequence  up  to  the 
last  level  and  decide,  with  the  help  of  a 
counselor,  whether  to  go  on  to  voca- 
tional training  or  college.  At  the  HMSC 
there  is  a similar  "in  house"  final  assess- 
ment at  the  end  of  the  ESOL  sequence 
that  leads  them  on  to  further  schooling. 

Programs  that  are  linked  to  voca- 
tional training  usually  are  required  to 
use  standardized  tests,  such  as  the 
BEST  or  TABE,  at  the  end  of  the  ESOL 
sequence  to  recommend  transition  to 
an  all-English  instruction  environment, 
although  ongoing  assessment  is  less 
formal.  At  Oficina  Hispana,  after  obtain- 
ing the  GED  in  Spanish,  students  must 
achieve  at  the  highest  level  of  the  TABE 
before  entering  the  job  training  pro- 
gram. Similarly,  at  the  Lao  Family 
Community  in  Milwaukee,  learners 


must  reach  the  4.0  grade  level  or  above 
as  determined  by  the  TABE  before  being 
referred  to  the  Milwaukee  Technical 
Area  College  for  job  training  or  higher 
education  studies. 

Conclusions 

It  is  clear  that  the  scope  of  a Native 
Language  Literacy  and  the  goals  of  the 
program  have  a great  influence  on  the 
type  of  assessment  process  used  by 
staff  to  recommend  transfer  or  program 
changes  in  the  program  of  study  of  its 
adult  learners.  The  first  language  of  the 

learners  has  imnlications  for  assessment 

A 

since  it  may  mean  having  access  to  the 
convenience  of  standardized  testing. 

For  programs  in  languages  other  than 
Spanish,  program  staff  are  forced  to 
create  unique  instruments  and  processes 
to  assess  the  attainment  of  crucial  levels 
of  reading  comprehension. 

Even  for  Spanish  literacy  programs, 
however,  consideration  of  the  cultural 
background  and  goals  of  the  students  is 
essential  to  provide  adequate  instruction 
and  assessment.  The  above  accounts  of 
the  assessment  methods  from  different 
programs  strongly  suggests  that  satisfac- 
tory progress  of  learners  in  ESOL  classes 
depends  greatly  on  whether  the  learner 
has  reached  at  least  a certain  level  of 
reading  comprehension  in  his/her 
native  langugage,  which,  in  terms  of 
elementary  school,  means  a sixth  grade 
level. 

This  aspect  of  the  process  of  second 
language  acquisition  among  adults 
echoes  the  findings  in  second  language 
acquisition  at  the  elementary  education 
level  that  indicate  that  children  without 
literacy  skills  in  their  native  language 
need  from  six  to  seven  years  of  school- 
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ing  in  the  native  language  before  being 
able  to  perform  adequately  in  an  L2 
instruction  environment  (Collier,  1992). 
Since  it  is  often  at  the  instance  of  adults 
eager  to  receive  ESOL  instruction  that 
early  transfers  and  misplacements  occur, 
this  finding  underscores  the  importance 
of  proper  counseling  and  placement 
while  attending  to  the  affective  needs 
of  the  learners. 

In  order  to  attend  to  the  needs  of 
adults  with  low-level  reading  compre- 
hension skills,  it  is  evident  that  Native 
Language  Literacy  instruction  along 
with  or  prior  to  ESOL  instruction  is  the 
ideal  solution.  It  is  also  necessary  to 
understand,  however,  that  the  progress 
of  those  adult  learners  in  ESOL  classes 
will  be  a function  of  the  reading  compre- 
hension level  in  the  native  language,  the 
method  and  content  of  instruction  in  the 
native  language,  and  the  level  of  literacy 
required  in  the  ESOL  classes. 

Although  Native  Language  Literacy 
instruction  offers  a viable  alternative 
to  attending  the  needs  of  learners  of 
English  as  a second  language  with 
limited  literacy  skills  in  their  mother 
tongue,  there  are  many  issues  still 
unresolved  within  that  growing  spe- 
cialty. One  is  the  ability  to  identify  and 
assist  learning  disabled  adults  and  those 
with  highly  developed  oral  literacy  skills 
but  lacking  the  rudiments  to  progress  in 
highly  decontextualized  learning  envi- 
ronments. In  addition,  it  seems  essential 
that  ESOL  and  ABE  instruction  be 
linked  to  these  programs  for  purposes 
of  adequate  placement,  instruction, 
and  transition.  Knowing  that  adequate 
services  are  not  available,  NLL  adults 
refrain  from  registering  at  ESOL  pro- 
grams and  remain  unaccounted  for.  The 
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estimates  of  how  large  of  a population  is 
in  need  of  NLL  instruction  are  therefore, 
at  best,  only  vague. 

It  is  my  hope  that  enough  practitio- 
ners in  Massachusetts  will  continue  the 
much-needed  sharing  of  ideas  and 
discussion  of  these  and  other  issues  to 
have  a significant  impact  on  assessment 
and  instruction. 
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ADULT  EDUCATION  CENTERS  OFFERING  NATIVE  LANGUAGE  LITERACY 


Massachusetts 
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76  Crest  Ave.  Chelsea,  MA  02 1 50 
Tel  (617)  228-2102. 

Contact:  Wilma  Bonaparte,  Dean,  Chelsea 
Campus. 

Community  Learning  Center 

19  Brookline  Street,  Cambridge,  MA 
02139. 

Tel  (617)  349-6363. 

Language:  Haitian  creole 
Contact:  Cecile  De  Mornay. 

Centro  Hispano  de  Chelsea 

248  Broadway,  Chelsea,  MA  02 1 50 
Tel  (617)  884-3238 
Contact:  Michelle  Brown. 

Centro  Presente 

54  Essex  Street,  Cambridge,  MA  02 1 39 
Tel  (617)  497-9080 

Contact:  Juan  Gonzalez,  Ed.D.,  Director 
of  Education. 

Community  Education  Project 

JUNTOS  program 

317  Main  Street,  Holyoke  MA  01046. 

Tel  (413)  538-5770. 

Contact:  Julie  Rapaport,  Executive  Director 


Haitian  Multi-Service  Center 

1 2 Bicknell  Street,  Dorchester,  MA 
02121 

Tel  (617)436-2848. 

Contact:  Jean  Marc  Jean  Baptiste, 
Executive  Director. 

Hampshire  County  Jail  and  House 
of  Correction 

205  Rocky  Hill  Road,  P.O.  Box  7000, 
Northampton,  MA  01061  -7000. 
Contact:  Connie  Cronin,  Director/ 

Nan  Walsh,  instructor. 

Harborside  Community  Center 

3 1 2 Border  Street,  East  Boston,  MA 
02128. 

Tel  (617)  491-3178 
Contact:  Paul  Trunnell,  Director. 

Hmong  Lao  Family  Foundation 
Association 

345  Main  Street,  Fitchburg,  MA  0 1 420. 
Contact:  Mr.  Kou  Yang,  (508)  342-1892. 

International  Institute  of 
Greater  Lawrence 

454  N.  Canal  Street,  Lawrence,  MA 
01840. 

Tel  (508)  687-0981. 

Contact:  Rafael  Abislaiman,  Director, 

' or  Jenny  Garcia,  instructor. 
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La  Alianza  Hispana 

409  Dudley  Street,  Roxbury,  MA  021 19. 
Tel  (617)  427-7175 
Contact:  Paul  Mullaney,  Director  of 
Education. 

Lawrence  Public  Schools  Adult 
Learning  Center 

243  So.  Broadway  St.,  Lawrence,  MA 
01843. 

Tel  (508)  975-5917. 

Contact:  Jeanne  O’Brien,  Director  of 
Education. 

Mujeres  Unidas  en  Accion 

1 534  Dorchester  Ave.,  MA  02 1 22. 

Tel  (617)  265-3015. 

Contact:  Argentina  Cuevas,  instructor. 

Oficina  Hispana 

125  Armory  Street,  Jamaica  Plain,  MA 
021 19. 

Tel.  (617)  929-1400 

Contact:  Jaime  Talero,  Executive  Director/ 
Lillian  Newton-Arango,  Director  of 
Intake. 

Puerto  Rican  Cultural  Center 

2345  Main  Street,  Springfield,  MA  01  107. 
Tel  (413)  737-7450 
Contact:  Anna  Rodriguez,  Director. 


Salem  Harbor  Community 
Development  Corp. 

50  Leavitt  Street,  Salem  MA  0 1 970 
Tel  (508)  745-8071. 

Contact:  Deborah  Mercier-Cuenca, 
instructor. 

OTHER  STATES  (described  in  M.Gillespie 
& Ballering,  E,  Eds.,  1 992) 

Casa  Aztlan.  Chicago,  Illinois. 

Contact:  Jose  Hunter. 

El  Barrio  Popular.  New  York,  New  York. 
Contact:  Klaudia  M.  Rivera/ 

Deidre  Freeman. 

El  Paso  Community  College  Literacy 
Education  Action  Program.  El  Paso, 
Texas.  Contact:  Carol  Clymer-Spradling/ 
Celia  Esparza. 

Lao  Family  Community,  Inc.  and 
Milwaukee  Area  Technical  College 
Community  Based  Organization. 

Milwaukee,  Wl.  Contact:  Douglas  Doua 
Vue. 

Triton  Community  College  Adult 
Continuing  Education.  New  Horizons 
Center,  Melrose  Park,  Illinois. 

Contact:  Lilia  Salazar-Holst. 
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The  Worker  Education  Program 
at  SEIU  Local  285  


The  Worker  Education  Program 
(WEP)  was  founded  more  than 
four  years  ago  to  build  educa- 
tional opportunities  for  SEIU  Local  285 
members  and  other  workers  at  area 
hospitals  in  Massachusetts.  In  January, 
1995  WEP  began  offering  ESL  and  ABE 
classes  at  six  hospitals  through  a 
National  Workplace  Literacy  grant  from 
the  federal  Department  of  Education 
and  administered  through  the  Massa- 
chusetts Department  of  Education. 
Students  in  these  classes  work  mostly  in 
low-end  service  jobs  — housekeeping, 
maintenance,  or  as  nursing  assistants. 
Students  in  the  ESL  classes  mostly 
want  to  improve  their  speaking  to  be 
better  understood,  and  to  work  on 
their  English  reading  and  writing. 

ABE  students  at  one  hospital  want  basic 
reading  and  writing  skills;  at  the  other 
two  sites,  students  seek  more  advanced 
communication,  presentation  and 
writing  skills.  Classes  are  scheduled 
based  on  when  workers  can  come  to 
class  (shifts),  not  skill  levels,  so  each 
class  has  a range  of  skills. 

WEP  teachers  respect  and  draw  on 
the  knowledge  and  creativity  that  adult 
learners  bring  to  the  classroom.  We  are 
developing  a participatory,  student- 
centered  curriculum  that  has  workers' 
interests  and  issues  at  its  core.  Through 
one-on-one,  small  group  and  whole  class 
activities,  teachers  and  students  work 
together  to  address  a wide  range  of 
issues  regarding  their  work  and  commu- 


nities. We  feel  that  education  at  the 
worksite  should  develop  workers' 
understanding  of  the  complex  forces  at 
their  workplace  and  give  them  tools  to 
deal  with  those  issues. 

How  we  came  up  with  the  tool 

We  started  out  wanting  to  develop 
a r>rp/nnst  asspssmpnt  tool  for  our  ESL 

— r — ' r 

and  ABE  classes  because  1)  we  as  a staff 
had  been  struggling  with  the  question 
of  systematizing  our  assessment  to 
adequately  measure  gains  in  our  pro- 
gram as  a whole,  and  2)  programs 
funded  by  the  National  Workplace 
Literacy  grant  must  use  placement  and 
pre/post  testing  tools.  The  DOE  report- 
ing forms  ask  for  "results"  from  pre- 
and  post-assessment;  you  check  off 
whether  your  program  is  using  stan- 
dardized tests  (CASAS,  TABE,  ABLE, 
etc.),  supervisor  ratings,  self-esteem 
inventories,  group  interviews,  job- 
related  skill  competency  tests,  or  any 
of  a myriad  of  tools. 

We  felt  it  would  make  sense  to 
develop  something  we  could  use  in  all 
of  the  classes.  This  would  "standardize" 
the  pre/ post  assessment  part  of  our 
program  and  make  reporting  easier.  We 
also  wanted  to  see  if  using  the  same  tool 
could  help  teachers  with  diverse  student 
groups  have  a common  beginning  and 
ending  point;  this  might  help  us  see 
how  students  are  progressing  across 
the  program. 

WEP  teachers  were  particularly 
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We  wanted  an 
assessment  tool 
that  would  pro- 
vide the  learners 
with  choices  so 
students  would 
have  some  control 
over  the  assess- 
ment process. 


interested  in  developing  an  "alterna- 
tive" assessment  tool.  We  feel  that 
standardized  tests  or  skill  competency 
tests  often  don't  capture  many  kinds  of 
learning.  They  use  decontextualized 
language,  assume  a certain  literacy  level, 
and  don't  reflect  how  students  may  be 
using  literacy  in  their  daily  lives. 

We  had  all  read  about,  experimented 
with  and  developed  alternative  assess- 
ment tools  in  previous  classes.  Some 
teachers  have  taken  ideas  from  Adven- 
tures in  Assessment  in  previous  classes, 
some  have  drawn  on  chapter  8,  "Evalua- 
tion: What  Counts  as  Progress?"  in  Elsa 
Aurbach's  Making  Meaning,  Making 
Change.  We  set  a meeting  date  to  dis- 
cuss the  prospective  tool. 

At  the  meeting,  we  all  talked  about 
what  we  wanted  our  tool  to  be  and  do. 

It  should  build  on  students'  strengths, 
not  show  them  what  they  can't  do.  We 
wanted  an  assessment  tool  that  would 
provide  the  learners  with  choices  so 
students  would  have  some  control  over 
the  assessment  process.  In  addition,  we 
wanted  a customized  tool  that  would 
measure  what  we're  actually  teaching  in 
our  classes:  basic  reading,  writing  and 
communication  skills  that  can  be  trans- 
ferred to  learners  work  and  home  lives. 
We  had  already  placed  students  in 
classes  with  a one-page  intake  interview 
and  a brief  reading/writing  sample,  and 
had  some  idea  of  what  workers  wanted 
to  focus  on.  We  also  felt  our  tool  should 
reflect  learners'  progress  in  affective 
areas  like  students  speaking  with  more 
confidence,  starting  to  use  language  in 
new  ways,  or  developing  new  writing 
strategies. 

Like  our  curriculum,  we  wanted  our 
assessment  to  be  workplace-oriented.  In 
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other  words,  we  wanted  it  to  measure 
changes  in  students'  language  use  at 
the  worksite  as  well  as  outside  work. 

We  decided  to  have  a three-compo- 
nent pre/post  assessment:  a reading 
sample,  a writing  sample  and  an  inter- 
view. The  reading  and  writing  samples 
would  capture  changes  in  the  basic  skills 
we  are  teaching,  the  interview  would 
get  at  changes  in  language  use  in  other 
areas.  One  group  of  teachers  put  to- 
gether a packet  of  readings,  comprehen- 
sion questions,  and  a writing  question; 
another  group  developed  the  interview. 

How  we  thought  the  tool  would  work 

Our  expectation  was  that  learners 
would  work  with  a reading  and  writing 
sample  at  the  beginning  and  end  of  the 
term.  Comparing  the  two  would  allow 
students  and  teachers  to  observe 
progress.  The  interview,  conducted  at 
the  beginning  and  end  between  teacher 
and  learner,  would  capture  changes  and 
progress  in  language  use  in  and  outside 
the  workplace. 

Reading  and  writing  samples: 
Learners  could  choose  a single  reading 
from  a packet  of  six  texts  of  varying 
difficulty  from  published  collections 
of  student  writings,  short  stories  and 
novels.  After  reading  the  piece  they'd 
chosen,  each  student  would  discuss  it 
with  the  teacher  using  a few  basic  compre- 
hension questions  (we  wrote  two  or  three 
questions  for  each  reading).  The  teacher 
would  write  down  the  student's  answers 
and  any  other  notes  on  reading  strate- 
gies, difficulties,  etc.  The  student  would 
then  write  a response  to  an  open-ended 
question:  When  you  read  this,  what  do  you 
think  about?  What  does  this  remind  you  of? 
How  does  this  compare  with  your  experience? 
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The  readings  were  all  about  people 
working;  they  included  a housekeeper's 
description  of  her  work,  an  immigrant 
describing  differences  in  working  in  her 
country  and  here,  etc.  We  hoped  learn- 
ers would  connect  to  these  themes 
enough  to  be  motivated  to  respond  in 
writing.  (And  we  wanted  the  writing 
sample  to  be  interesting  and  meaningful!) 

When  choosing  texts  for  the  packet, 
we  found  it  difficult  to  pick  readings 
that  would  cover  the  range  of  basic  ESL 
through  "advanced"  ABE.  ABE  and  ESL 
teachers  also  had  different  ideas  about 
which  Questions  to  ssk.  for  rccidinj 
comprehension  (what  to  look  for).  We 
wondered  if  the  packet  would  be 
equally  useful  for  both  ABE  and  ESL 
teachers  and  students. 

Interview:  The  interview  included 
questions  about  students'  prior  school- 
ing, where  and  when  they  read,  write  or 
speak  English,  when  they  use  their 
native  language  (for  ESL  students),  and 
goals  for  the  cycle. 

What  actually  happened 
Pre-assessment  pros 

1.  Time:  Since  the  cycle  had  already 
started  by  the  time  we  developed  them, 
teachers  did  the  pre-assessment  in  class. 
Each  class  handled  the  activities  differ- 
ently. Some  teachers  took  students  out 
of  the  class  one  by  one  while  the  rest  of 
the  class  worked  on  another  assignment; 
others  had  learners  all  working  on 
reading  and  writing  at  the  same  time, 
helping  each  student  as  s/he  was  ready 
to  discuss  the  reading  and  then  write. 
Most  teachers  took  a whole  class  (two 
hours)  to  do  the  reading  and  writing 
samples  and  needed  another  class  to 
do  interviews. 


2.  Choice:  students  could  choose  the 
reading  selection  they  were  most  com- 
fortable with;  this  relaxed  them  and  may 
have  resulted  in  more  "productive" 
reading  and  writing.  If  a learner  chooses 
a reading  obviously  too  "easy"  for  her/ 
his  level,  teachers  could  do  that  reading 
with  them  but  also  suggest  they  try  a 
more  difficult  passage. 

Pre-assessment  cons 

1.  Logistics:  Some  teachers  found  it 
difficult  to  juggle  all  the  individual 
attention  needed  during  the  reading 
and  writing  samples.  One  teacher,  for 
example,  said  it  was  hard  to  focus  on  the 
student  with  whom  she  was  working, 
knowing  other  students  were  finishing 
up  the  reading,  having  trouble  with  it, 
or  waiting  to  start  writing.  Other  teach- 
ers felt  that  the  reading,  comprehension 
and  writing  activities  took  too  much 
time. 

2.  Levels:  Even  though  the  reading 
selection  included  a few  very  short, 
basic  pieces  (one  with  just  one  sentence), 
there  were  still  students  who  were  at 

a very  basic  literacy  level.  For  these 
students,  the  tool  didn't  really  capture 
what  they  could  do.  For  the  more  ad- 
vanced ABE  students,  the  readings  and 
basic  comprehension  questions  were 
easy,  and  did  not  always  measure  their 
ability  to  use  more  advanced  reading 
strategies. 

3.  Interview/timing:  The  interview 
turned  out  to  be  cumbersome  and 
repetitive.  Some  questions  repeated 
information  asked  during  intake, 
others  had  already  come  up  and  been 
addressed  during  previous  goal-setting 
activities.  Some  instructors,  not  wanting 
to  pull  individual  students  out  one  by 
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one,  turned  the  interview  into  a whole 
class  activity.  Two  teachers,  for  example, 
posted  interview  questions  on  newsprint 
for  discussion  and  made  class  charts 
with  students'  responses.  Using  discus- 
sion helped  students;  they  could  hear 
and  reflect  on  other  people's  experi- 
ences, and  then  talked  about  their  own 
more  fully  than  some  students  who  did 
the  interview  alone  with  the  teacher.  A 
few  teachers  who  did  individual  inter- 
views ended  up  completing  them  over 
the  course  of  two  or  three  classes. 

Post-assessment  pros 

1.  Familiarity:  Because  students  had 
already  done  the  activities  at  the  begin- 
ning of  the  cycle,  students  were  familiar 
with  them  and  could  work  comfortably. 
Teachers  found  it  easier  to  juggle  the 
reading  and  writing  during  class. 

2.  Assessing  change:  For  some  teachers 
and  students,  doing  the  same  activities 
and  comparing  the  two  sets  of  samples 
proved  useful  in  assessing  changes  in 
reading  and  writing  strategies  during 
the  term.  In  one  ESL  class,  for  example, 
students  talked  about  how  they  felt 
more  comfortable  with  reading,  and 
noted  that  they  were  able  to  write  much 
more  independently  than  at  the  begin- 
ning of  the  term. 

Post-assessment  cons 

1.  Choice:  Some  teachers  reported  that 
students  chose  the  same  reading  selec- 
tion they  had  done  during  the  pre- 
assessment. While  doing  the  same 
selection  can  show  changes  (teachers 
expected  this  might  happen),  it  can  also 
mislead.  Some  students,  for  example, 
may  have  remembered  the  discussion 
from  the  pre-assessment,  and  relied  on 
that  rather  than  the  text  to  do  the  read- 
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ing  comprehension  questions.  In  some 
cases,  teachers  let  students  do  the  same 
selection  and  then  also  asked  them  to 
choose  another. 

Other  students  chose  an  "easier" 
passage  than  the  one  they  had  done  for 
the  pre-assessment.  Even  though  teach- 
ers could  encourage  students  to  pick 
a "harder"  piece,  they  felt  it  necessary 
to  allow  student  choice.  Teachers  then 
found  themselves  asking  how  to  inter- 
pret a student  choosing  an  easier  read- 
ing. It  could  be  because  the  student  was 
tired,  felt  rushed,  or  wanted  to  show 
improvement  and  therefore  went  for  a 
"safe"  choice.  But  we  didn't  really  know 
the  reasons  for  students  choosing  as 
they  did.  Sometimes  it  was  hard  to 
distinguish  between  what  students  have 
remembered  and  internalized  from  the 
pre-assessment,  and  what  new  progress 
they've  made  when  they  do  the  post- 
assessment. For  some  students,  remem- 
bering and  internalizing  is  a sign  of 
progress. 

Some  reflections 

We  set  out  to  develop  a "standard" 
tool  we  could  use  for  ESL  and  ABE  at  all 
of  our  sites.  But  we  also  wanted  it  to  be 
flexible,  so  teachers  could  adapt  it  to 
meet  their  teaching  styles  and  students' 
needs.  Teachers  did  adapt  it,  maybe  to 
the  point  that  it  was  no  longer  very 
standard.  Also,  ABE  teachers  found  that 
they  wanted  to  measure  skills  (reading 
and  writing)  strategies  that  the  reading 
and  writing  samples  didn't  capture.  ESL 
teachers  with  beginning  speakers  found 
the  interview  (designed  to  reflect  affec- 
tive areas  and  language  use),  somewhat 
useless  (a  few  teachers  conducted 
interviews  in  students'  native  languages 


622 


Volume  8:  December  1995 


when  they  could).  Our  program  may 
need  to  develop  separate  assessment 
tools  for  ABE  and  ESL  classes. 

Most  importantly,  we  wanted  assess- 
ment activities  that  would  help  students 
see  their  progress.  Comparing  reading 


and  writing  samples  helped  some 
students  see  areas  they'd  improved. 
Many  students  need  to  learn  how  to 
see  and  evaluate  their  progress;  this 
is  a process  that  takes  time. 
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Bringing  Learners  into  Goal-Setting 
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Anne  Burke, 
Linda  Gosselin, 
Jane  Shea 


Quinsigamond 
Community  College 
Worcester,  MA 


Quinsigamond  Community 
College  has  several  adult 
education  programs,  one  on 
campus  at  night  and  several 
during  the  morning  at  off-campus  sites 
in  Worcester  and  Southbridge.  Our 
program  holds  classes  in  the  morning  at 
St.  Paul's  School.  We  offer  ASE,  ABE, 
and  ESL. 

Our  adult  ESL  students  represent  a 
cross-section  of  the  immigrant  commu- 
nity in  Worcester.  The  largest  segment 
of  our  student  population  is  Spanish- 
speaking, with  the  majority  from  Puerto 
Rico  and  the  rest  from  various  Central 
and  South  American  countries.  They 
make  up  60-70%  of  our  student  enroll- 
ment. The  remainder  of  the  students  are 
from  a variety  of  countries  in  Asia,  the 
Mideast,  and  Eastern  Europe  and  Russia. 

Our  day  students  are  predominantly 
female  (80%).  Some  are  employed  or  are 
on  public  assistance.  Many  are  house- 
wives. Some  are  single  parents;  others 
are  single  and  never  married.  Many  of 
the  male  students  work  at  night  and 
seem  evenly  divided  between  being 
married  or  single.  Ages  range  from  18- 
70. 

The  program  seeks  to  serve  the 
needs  of  our  students,  whether  they  are 
seeking  better  employment,  hoping  to 
pursue  higher  education,  or  just  coming 
to  enrich  their  own  language  skills. 

The  program  is  an  open  enrollment, 
open  ended  one.  Each  of  the  three 
classes  must  maintain  a class  enrollment 


of  13  students.  As  students  reach  their 
goals  or  withdraw  early  from  the  pro- 
gram, new  students  are  accepted  into 
the  program  from  a waiting  list. 

Working  in  the  Community 

Our  community-based  program 
serves  adults  with  limited  English 
proficiency  who  desire  to  maintain  and 
improve  their  quality  of  life  in  American 
society.  Our  goal  is  to  enable  adult 
learners  to  actively  formulate  their  own 
educational  goals. 

This  goal  is  attained  through  a 
variety  of  ways.  When  students  join  a 
class,  they  complete  an  IEP  (individual 
education  plan),  identifying  for  instruc- 
tors and  themselves  the  learners'  educa- 
tion goals.  At  the  start  of  each  curricu- 
lum unit  of  study,  students  and  their 
instructor  decide  on  which  particular 
areas  they  want  to  work,  via  a needs 
assessment/interest  profile.  Together 
they  adapt  the  curriculum  to  class  needs. 

The  structure  of  our  ESL  program 
is  based  on  the  MELT  (Mainstream 
English  Language  Training)  student  per- 
formance levels,  adapted  to  our  own  three 
proficiency  levels.  Level  1 includes  SPL 
0-2;  Level  2 consists  of  SPL  3 and  4;  Level 
3 embraces  SPL  5 and  6.  Students  may 
move  to  higher  levels  as  they  complete 
their  goals  and  are  assessed  to  be  ready. 

Students  meet  nine  hours  a week  for 
38  weeks  each  year.  They  can  continue 
in  the  program  until  they  complete  all 
three  levels. 
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This  tri-level  program  design  was 
created  in  1991.  Before  that,  each  in- 
structor worked  independently,  cover- 
ing life  skills  and  grammar  randomly. 
This  program  weakness  often  led  to 
repetition  of  some  life  skill  materials 
and  omission  of  others.  It  was  difficult 
to  assess  the  readiness  of  a student  to 
progress  to  the  next  level.  To  improve 
our  program  and  to  better  meet  the 
needs  of  our  students,  we  decided  a 
more  formal  curriculum  was  needed. 

In  1991,  we  three  ESL  instructors 
won  approval  to  use  program  develop- 
ment funds  to  create  a curriculum  that 
would  provide  better  consistency  of  skill 
development  from  one  level  to  the  next. 

Working  together  throughout  the 
summer  of  1991,  we  designed  guidelines 
for  life  skill  competencies,  grammar,  and 
vocabulary  appropriate  to  each  level, 
flexible  enough  to  meet  our  instructional 
needs  and  the  individual  educational 
plans  of  our  students. 

Assessment  Development 

After  utilizing  the  curriculum  for 
one  year,  we  determined  that  we  needed 
a more  formal  instrument  for  assessing 
a student's  progress  and  readiness  to 
move  on  to  the  next  level. 

Our  initial  means  of  assessing 
student  progress  within  this  curriculum 
was  very  subjective.  Every  ten  weeks  the 
teacher  would  write  a narrative  concern- 
ing each  student's  general  progress  in 
reading,  writing,  speaking,  and  listening. 

This  means  of  assessment  did  not 
really  provide  us  the  information  we 
needed.  It  didn't  inform  us  enough  as  to 
a student's  progress  in  specific  life  skills 
in  class  nor  which  skills  were  required 
to  review  or  to  further  study  in  class. 
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Therefore,  in  the  summer  of  1992, 
again  with  program  development  funds, 
we  collaborated  on  improving  the  assess- 
ment system.  We  started  by  examining 
various  articles  from  Adventures  in 
Assessment  to  see  what  other  programs 
were  using  to  evaluate  student  progress. 

In  particular  we  found  "Three  by 
Three  by  Four:  Ongoing  Assessment  at 
the  Community  Learning  Center,"  by 
Karen  Ebbitt,  Priscilla  Lee,  Pam  Nelson, 
and  Joann  Wheeler  in  Volume  2 of 
Adventures  in  Assessment  helpful.  Using 
it  as  a guide  we  decided  that  our  assess- 
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assumed  topics  to  be  covered,  but  would 
be  a form  that  would  allow  the  instruc- 
tor and  the  learners  to  review  and  list 
the  skills  and  materials  presented  in  class. 

Our  decision  was  based  on  our 
philosophy  of  encouraging  our  learners 
to  fully  participate  in  their  own  assess- 
ment. Because  it  was  important  to  have 
some  way  of  evaluating  progress  from 
the  teacher's,  as  well  as  the  student's, 
point  of  view,  we  developed  two  forms: 
a Student  Self-Evaluation  Form  and  a 
Teacher  Assessment  Form  (see  Forms  A 
and  B,  pp.  39-40). 

On  the  Student  Self-Evaluation  Form 
we  wanted  to  use  vocabulary  that  could 
be  easily  understood  by  most  students. 
We  decided  to  have  students  assess  how 
they  felt  about  their  progress  on  compe- 
tencies worked  on  in  class.  We  thought 
it  would  be  easier  for  lower  level  stu- 
dents to  check-off  how  they  felt  about 
their  progress,  rather  than  writing  about 
what  specific  progress  they  might  have 
made.  For  more  advanced  students,  we 
provided  space  so  they  could  write 
about  their  progress  in  specific  skill 
areas. 


We  decided  that 
our  assessment 
would  not  include 
a listing  of 
assumed  topics 
to  be  covered,  but 
would  be  a form 
that  would  allow 
the  instructor  and 
the  learners  to 
review  and  list 
the  skills  and 
materials 
presented  in  class. 
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Assessment  Format 

The  Student  Self-Evaluation  Form 
is  completed  through  a brainstorming 
session  in  which  students  verbalize, 
with  or  without  the  teacher's  help,  the 
skills  and  materials  covered  over  a 
particular  period  of  time.  Brainstorming 
offers  the  opportunity  to  review  and 
recognize  what  was  accomplished  in 
class.  The  list  of  skills  generated  is 
written  on  the  board  and  copied  by  the 
students  on  to  their  forms.  For  example, 
if  we  had  worked  on  health  and  medi- 
cine the  list  of  skills  generated  might  be: 

1.  Review  of  parts  of  the  body. 

2.  Review  of  simple  illnesses. 

3.  Other  common  illnesses. 

4.  Describing  one's  symptoms. 

5.  Calling  to  make  a doctor's  appoint- 
ment. 

6.  Filling  out  a medical  history  form. 

7.  Understanding  medicine  labels. 

8.  Common  medical  tests  and  proce- 
dures. 

While  the  students  assess  their  own 
progress  in  these  competencies,  the 
teacher  records  the  same  skills  on  a 
Teacher  Assessment  Form  for  each 
student  as  a separate  assessment.  As 
soon  as  possible  after  completion  of  both 
forms,  student/ teacher  conferences  are 
scheduled  during  class  time  to  discuss 
and  compare  evaluations.  The  students 
and  teacher  use  the  forms  to  identify 
those  items  that  have  been  mastered 
and  those  that  require  further  study. 

The  conference  allows  discussion  of 
any  other  difficulties  with  the  unit  or 
other  factors  affecting  the  student's 
learning  that  may  not  have  come  to  light 
previously.  The  conference  is  important 
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so  that  the  instructor  and  learner  have  a 
mutual  understanding  of  progress  made 
and  obstacles  to  learning.  If  there  is  a 
discrepancy  between  what  progress 
the  teacher  and  student  think  has  been 
made,  discussion  follows.  More  often 
than  not,  the  gaps  occur  because  the 
student  is  not  very  confident  in  his/her 
ability.  It  is  then  the  task  of  the  teacher 
to  remind  the  student  of  the  successes 
he  /she  had  in  the  classroom  and  what 
the  student  could  do  at  the  beginning 
and  what  the  student  can  do  better  now. 

This  can  be  accomplished  through 
telling  the  student  what  the  teacher  has 
observed  or  by  showing  the  student 
samples  of  classwork  done  that  demon- 
strate progress. 

Reflections 

Preparing  for  this  article  has  encour- 
aged us  to  formally  evaluate  our  use  of 
this  on-going  assessment  and  come  up 
with  the  following  list  of  strengths  and 
weaknesses: 

Strengths 

1.  The  format  is  flexible  enough  to  be 
used  at  any  point  in  the  program. 
Because  students  are  only  assessed  on 
the  materials  covered  from  the  time 
they  enter  the  program,  forms  better 
serve  our  open-entry,  open-exit 
program  than  standardized  instru- 
ments. Standardized  tests  may  test 
students  on  materials  not  covered 

in  class.  They  would  not  give  us  the 
information  we  need  in  order  to 
assess  student  progress  and  readiness 
to  advance  to  the  next  level. 

2.  Student  involvement  in  reflecting  on 
skills  and  materials  covered  leads  to 
their  recognition  that  learning  has 
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taken  place. 

3.  A blank  assessment  form  allows  for 
teacher  flexibility  in  adapting  the 
curriculum  to  the  needs  of  a particu- 
lar group  of  students  at  each  class 
level.  This  is  important  because  what 
is  covered  in  a particular  unit  of  study 
can  vary  from  year  to  year. 

4.  Use  of  a student  form  and  a teacher 
form  provides  a balanced  record  to 
support  the  determination  that  a 
student  is  ready  to  move  on  to  a 
new  level.  The  higher  level  teacher 
can  see  the  units  covered,  as  well  as 
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nesses.  Because  evaluation  includes 
a self-assessment,  the  teacher  also 
gains  a sense  of  the  student's  own 
awareness  of  competency. 

5.  Instructors  can  use  the  instrument 
to  identify  areas  that  need  further 
review  or  instruction.  For  example,  if 
the  majority  of  students  respond  "not 
good"  to  completing  a job  application, 
the  instructor  knows  that  further 
work  in  that  area  is  necessary. 

Weaknesses 

1.  Often  two  or  more  class  days  are 
required  in  order  to  complete  the 
assessments  and  follow-up  confer- 
ences. This  time  must  be  seen  as 
necessary  and  a learning  experience 
of  its  own  value  for  the  students. 

2.  Students  tend  to  want  to  evaluate 
their  ability,  not  their  progress.  In  the 
theme  of  'health/  for  example,  the 
students  may  have  worked  on  calling 
to  make  a doctor's  appointment. 
When  it  comes  time  to  evaluate  their 
ability  to  do  this,  students  may  tend 
to  want  to  evaluate  themselves  as 


compared  to  a sample  conversation 
used  in  class.  It  can  be  difficult  to  get 
them  to  evaluate  themselves  or  their 
progress  from  the  time  they  started 
and  what  they  can  now  communicate. 

CHANGES 

Our  on-going  assessment  tools  are 
an  effective  means  of  assisting  learners 
to  evaluate  their  own  progress  and  pro- 
viding teachers  with  the  documentation 
needed  to  determine  a student's  readi- 
ness to  progress  through  the  levels.  How- 
ever, we  have  seen  a need  to  make  some 
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ment  to  better  meet  our  program  needs: 

1.  Frequency  of  Use 

The  forms  were  originally  intended 
to  be  administered  every  ten  weeks, 
according  to  DOE  guidelines.  How- 
ever, we  found  it  more  logical  for 
assessment  to  take  place  after  major 
units  of  study.  This  schedule  is  still  in 
compliance  with  DOE  expectations, 
while  more  in  line  with  our  own 
program  and  learner  needs.  The 
length  of  time  between  evaluations 
can  vary  according  to  how  long  it 
takes  to  cover  a theme  for  a particular 
group  of  students. 

2.  Form  Changes 

A.  Student  Self-Assessment  Form 
We  plan  to  add  a section  to  the  form 
for  students  to  assess  their  attendance 
and  participation. 

B.  Teacher  Assessment  Form 
We  have  deleted  the  evaluation  of 
attitude  and  effort  as  they  are  already 
reflected  in  attendance  and  participa- 
tion. Also,  trying  to  evaluate  these 
factors  separately  can  be  very  subjec- 


ERiC 


fifing  8:  December  1995 


tri 


Because  evalua- 
tion includes  a 
self-assessment, 
the  teacher  also 
gains  a sense  of 
the  student's  own 
awareness  of 
competency. 
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tive.  At  times  there  can  be  personality 
clashes  between  a student  and 
teacher.  When  it  is  time  to  evaluate 
a student's  attitude  and  effort,  these 
negatives  may  unconsciously  influ- 
ence a teacher's  perception  of  these 
factors.  A student's  attendance  can 
be  quantified.  If  there  are  no  factors 
influencing  attendance,  good  atten- 
dance reflects  good  attitude  toward 
learning.  Observations  of  a student's 
active  participation  in  class  activities 
is  evidence  of  effort  to  make  progress. 


CONCLUSION 

Writing  this  article  gave  us  the 
opportunity  to  reflect  on  our  on-going 
assessment,  identify  its  strengths  and 
weaknesses,  and  decide  what  changes 
should  be  made  to  improve  our  assess- 
ment process. 

Overall,  we  are  pleased  with  the 
on-going  assessment  tools  we  use. 
They  have  served  our  needs  well  since 
we  began  designing  them  in  1991. 

With  the  changes  we  are  making 
in  our  on-going  assessment  tools,  we 
are  confident  that  they  will  continue 
to  serve  the  needs  of  our  program  into 
the  future. 
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FORM  A 

Teacher  Assessment  Report 

STUDENT DATE  

UNIT INSTRUCTOR  


Excellent  Very  Good  Good  Not  Good 

Attendance  

Participation  

Attitude  

Effort  

I . Competencies  Can  do  Needs  Work  Can  t do 


1. 

2. 

3. 

4. 

* 

5. 

6. 

7. 

8. 

9. 

10. 

II.  Grammar  Structures 

1. 

2. 

3. 

4. 

5. 

III.  Other  skill  areas  (strengths  & weaknesses,  plus  factors  affecting  learning) 
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FORM  B 

Student  Self-Assessment  Report 

NAME DATE  

UNIT TEACHER  


I . How  do  you  feel  about  Very  Good  Good  Not  Good  Bad 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

How  do  you  feel  about  your  general  progress  in  this  unit? 

II.  Skill  Areas.  How  do  you  feel  about  your  progress  in  these  things? 


1 . Speaking  and  Listening 

2.  Reading 

3.  Writing 

4.  Grammar 

III.  General  Comments 
I . Now  I can  do  these  things  better  in  English 


2.  I am  still  having  problems  with  these  things  in  English 
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Learning  from  Experience 

Do  ESOL  Students  Want 
to  be  Tested  More  Often? 


I am  presently  a worksite  ESOL 
teacher  for  Jewish  Vocational  Ser- 
vices which  provides  on-site  ESOL 
instruction  for  employees  of  Beth  Israel 
and  Children's  Hospitals  at  all  levels 
(from  literacy  to  advanced). 

When  I was  first  approached  about 
getting  my  students'  input  on  assess- 
ment I was  rather  hesitant,  uncertain 
if  my  students  would  want  to  have  a 
discussion  on  assessment  or  want  their 
feelings  put  on  tape  and  print. 

I decided  to  ask  my  Beginner  2 class 
(the  highest  level  I taught  at  this  time) 
if  they  would  be  willing  to  share  their 
views.  The  class  is  made  up  of  eight 
students  from  Central  and  South 
America,  all  of  whom  have  had  at  least 
eight  years  of  formal  education  in  their 
countries.  Most  of  the  students  work  in 
the  housekeeping  department,  others  in 
Central  Processing  and  in  the  Laundry 
department. 

The  class  meets  for  an  hour  three 
times  a week.  The  emphasis  is  on  work- 
related  language  and  on  strengthening 
communicative  skills.  All  students  in  the 
Beginner  2 class  have  fairly  good  aural/ 
oral  skills  but  lack  basic  structures  and 
spelling  to  write  correct  and  indepen- 
dently. 

The  class  is  highly  motivated,  and 
students  give  honest  feedback  on  how 
they  felt  the  class  went  and  what  they 
felt  they  needed  for  subsequent  classes. 

I have  taught  the  same  group  for  over 
a year. 


I took  a copy  of  Adventures  in  Assess- 
ment into  class  and  briefly  explained 
what  the  journal  was  about,  read  the 
letter  from  the  editor  and  then  asked  if 
they  would  be  willing  to  share  their 
opinions  of  assessment. 

I was  quite  surprised  by  their  over- 
whelmingly positive  response.  I thought 
that  they  would  not  be  interested  at  all. 

I now  began  to  look  on  the  project  as  a 
learning  experience  for  me.  Like  many 
other  ESOL  teachers  I often  wrestle  with 
thoughts  on  assessment;  Should  I be 
testing  students  more  often?  Do  stu- 
dents want  to  be  tested?  Would  they  be 
discouraged  if  they  did  not  do  so  well? 
How  much  class  time  should  I take  for 
testing?  What  kinds  of  tests  would 
measure  their  progress  accurately/ 
effectively  and  help  students  evaluate 
their  own  progress? 

I have  also  felt  a reluctance  to  test 
due  to  the  lack  of  effective  tests/ tools 
that  evaluate  increases  in  levels  of 
aural /oral  skills  and  that  capture  the 
advances  of  the  most  limited  of  the 
ESOL  learners.  At  the  workplace,  issues 
and  questions  of  assessment  are  more 
evident  since  students'  supervisors  (and 
the  students  themselves)  often  would 
like  to  see  more  concrete  evidence  of 
progress.  Other  than  the  initial  intake 
and  placement  tests,  no  formal  tests  are 
conducted  in  class.  Level  changes  and 
end-of-the-cycle  evaluations  are  based 
on  classroom  observations  and  indi- 
vidual teachers  assessment  tools  like 
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My  own  hesita- 
tion in  not  testing 
students  was 
partly  so  as  not  to 
burden  them  with 
added  stress  that 
I often  felt  when 
being  tested. 


observations,  on-going  oral  feedback 
activities,  student  writings,  participation 
and  attendance. 

I went  into  my  class  the  next  day 
with  my  tape  player.  I handed  out  a list 
of  questions  and  gave  students  a few 
minutes  to  look  them  over.  I then  pro- 
ceeded to  record  the  resulting  discus- 
sion. Each  student  seemed  eager  to  say 
a few  words  for  every  question.  It  was 
obvious  this  was  a topic  they  wanted  to 
discuss. 

First  we  talked  about  tests  in  gen- 
eral. Students  related  their  experiences 
with  driver's  license,  citizenship  and,  of 
course,  medical  tests.  They  talked  about 
tests  in  their  native  countries:  to  go  from 
one  grade  to  the  next,  for  jobs  where  the 
applicant  who  scored  the  highest  would 
get  the  job.  They  did  not,  however,  have 
much  experience  with  formal /informal 
English  language  tests. 

We  then  went  back  to  the  questions. 
Some  students  chose  to  use  this  as  a 
writing  assignment;  others  just  gave 
their  answers  orally. 

Students  said  tests  were  very  impor- 
tant. They  said  tests  would  enable  stu- 
dents as  well  as  teachers  to  see  whether 
and  how  much  a student  learned  or 
progressed.  One  student  said,  "Tests 
show  you  what  you  learned  and  where 
you  need  help."  Another  student 
pointed  out  that  it  is  not  enough  to  have 
classes  all  the  time.  "A  test  every  once 
in  a while  is  important  to  see  if  one  is 
learning.  If  you  study  something  you're 
supposed  to  know  if  you  are  learning 
something.  Everyday  class  class  class... 
then  one  day,  a test.  It  is  important  to 
see  if  you  are  learning." 

Another  student  said  that  she 
learned  best  from  mistakes  and  that  she 
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would  learn  from  the  mistakes  she  made 
on  a test.  She  likened  testing  to  a new 
job  where  one  would  try  to  avoid  mak- 
ing the  same  mistakes. 

What  surprised  me  most  was  their 
response  to  the  question  'Do  you  like 
being  tested?'  All  but  one  student  said 
yes.  One  student  remarked  that  the  only 
kind  of  test  that  would  make  them  very 
nervous  would  be  one  that  would  make 
them  lose  a job  or  fail.  Students  seemed 
surprised  when  I admitted  that  even 
though  I knew  the  benefits  of  tests,  I 
myself  never  liked  any  kind  of  testing. 
Hence,  my  own  hesitation  in  not  testing 
students  was  partly  so  as  not  to  burden 
them  with  added  stress  that  I often  felt 
when  being  tested. 

After  some  investigation  the  student 
who  said  she  disliked  tests  was  one  who 
often  got  nervous  around  print  (even 
though  her  skills  were  above  average). 

It  seemed  important  to  this  student  that 
she  was  told  what  the  test  was  about 
and  that  the  instructions  and  format  be 
clear  and  simple.  "I  don't  like  tests  when 
I don't  know  what  I am  supposed  to  do. 
Sometimes  I don't  understand  the 
instructions."  She  also  remarked  that 
she  would  like  tests  more  when  she  has 
learned  more  grammar.  Is  this  because 
the  assessment  tools  to  which  she  was 
accustomed  tested  grammar /structure, 
not  communicative  competence? 

Most  students  indicated  a preference 
for  surprise  tests  every  once  in  a while. 
They  also  think  that  at  every  cycle  end 
(a  cycle  is  usually  anywhere  from  10-12 
weeks)  there  should  be  a final  exam- 
one  that  would  test  all  skills,  as  well  as 
retention  of  vocabulary  and  content 
covered.  For  this  final  exam,  students 
said  they  want  the  teacher  to  let  them 
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know  in  advance  when  they  would 
be  tested  and  that  the  teacher  should 
prepare  the  students. 

As  a result  of  this  project  I have 
promised  to  test  my  students  more 
often.  I would  like  to  spend  more  time 
and  energy  developing  assessment  tools 
for  my  classes  that  not  only  help  me  to 
evaluate  their  progress  but  also  help 
students  to  evaluate  their  own  progress 
and  build  on  their  strengths.  As  I de- 
ft 


velop  these  tools,  I need  to  consider  the 
concern  of  the  students  that  instructions 
and  expectations  of  the  assessment 
process  are  clear. 

We  can  get  students  in  the  dialogue 
by  asking  them  about  assessment/ 
testing  and  involve  them  in  the  analysis 
of  existing  standardized  and  alternative 
tests  and  in  the  development  of  testing 
formats  that  address  some  of  the  issues 
raised  by  the  students  in  this  class. 
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Letter 

What  Does  It  Mean  to  Get  an 
Authentic  Student  Perspective? 


We  all  know  it  is  difficult  to  get 
an  authentic  student  perspec- 
tive. However,  I believe  we 
all  must  try  to  incorporate  this  as  a goal 
in  thinking  about  adult  basic  education. 
The  more  we  ask  students  and  listen  to 
their  opinions,  the  more  effective  we 
will  be  in  the  classroom,  and  in  making 
policy  decisions  that  affect  adult  stu- 
dents and  the  services  provided  for 
them.  Currently,  programs  are  actively 
involving  students  in  writing  for  student 
journals,  being  members  of  advisory 
boards,  participating  in  student  leader- 
ship mini-grants,  etc. 

I want  to  share  my  experience  as  a 
person  in  the  field  who  is  involved  in  a 
project  that  is  striving  to  get  an  "authen- 
tic student  perspective."  The  purpose  of 
this  letter  is  not  to  defend  this  project, 
but  rather  to  look  at  and  reflect  the 
process  of  obtaining  a statewide  student 
perspective. 


by 

Michele  Verni 


World  Education 
SABES  Adult  Literacy 
Hotline  Coordinator/ 
Student  Leadership 
Coordinator 


Learner's  Voice  in  Public  Policy 
It  all  started  at  a World  Education 
retreat.  We  had  a small  group  discussion 
that  led  to  the  idea  of  getting  the 
learner's  voice  in  public  policy.  After 
the  retreat  three  of  us  met  to  discuss 
this  idea  further.  We  wanted  to  find  a 
vehicle  that  puts  the  "learner's  voice"  up 
front  in  the  policymaking  process.  We 
wanted  lawmakers  to  better  understand 
the  adult  learner  and  how  adult  basic 
education  (ABE)  programs  have  affected 
their  lives. 
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To  find  the  authentic  learner  voice, 
we  knew  adult  learners  had  to  be  in- 
volved at  the  start  and  in  fact,  they 
were.  The  first  thing  we  did  was  gather 
a group  of  five  students  for  a meeting. 
The  purpose  of  the  meeting  was  to 
discuss  the  idea  of  sending  a survey 
and  how  the  survey  questions  should 
be  formulated.  This  committee  of  adult 
students  was  devised  to  provide  a cross- 
section  of  adult  students  from  different 
programs.  Four  of  these  students  were 
involved  in  the  student  leadership  mini 
grant  projects  and  the  fifth  is  a Massa- 
chusetts student  delegate  who  has  been 
very  committed  to  student  leadership 
activities. 

At  the  meeting  we  asked  the  stu- 
dents, "How  has  your  learning  experi- 
ence affected  your  life?"  Each  person 
spoke  openly  about  their  educational 
experience,  its  rewards  and  obstacles. 
Everyone  shared  a personal  story  about 
his  or  her  involvement  with  adult  basic 
education  and  how  it  has  affected  their 
lives.  From  the  ensuing  discussion  we 
designed  a survey,  using  the  students' 
words  for  the  core  phrasing  of  the 
survey  questions. 

Our  next  challenge  was  to  distribute 
this  survey  to  students  statewide.  There 
are  five  SABES  (System  for  Adult  Basic 
Education  Support)  regional  centers  that 
correspond  to  the  geographical  regions 
of  Massachusetts.  First  we  contacted  all 
five  SABES  Regional  Coordinators  for 
names  of  programs  and  teachers  that 
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might  be  interested  in  our  project.  When 
we  called  these  programs,  everyone 
enthusiastically  agreed  to  administer 
and  collect  the  surveys.  We  sent  each 
program  an  instruction  sheet,  self- 
addressed  & stamped  envelopes  and  the 
number  of  surveys  they  requested.  We 
sent  approximately  800  surveys  to  about 
30  programs  throughout  Massachusetts, 
including  ABE,  GED,  ESOL  (English 
Speakers  of  Other  Languages),  one  on 
one  tutoring,  homeless  education, 
corrections  education,  etc. 

We  were  surprised  by  the  tremen- 
dous response  of  642  completed  sur- 
veys. We  think  we  had  such  a high  rate 
of  return  because:  1)  we  asked  that 
surveys  be  completed  in  May  or  June,  a 
natural  period  of  reflection  or  evaluation 
for  programs,  2)  we  sent  surveys  only 
to  programs  that  agreed  to  fill  out  the 
survey,  and  3)  students  developed  the 
content  of  the  survey.  An  unanticipated 
outcome  was  that  programs  asked  to 
keep  the  survey  for  a future  evaluation 
tool.  Overall,  the  distribution  process  of 
the  surveys  ran  smoothly  and  was  rela- 
tively low  cost  for  a statewide  project. 

We  have  hired  a graduate  student  to 
compile  the  data  and  assist  in  the  analy- 
sis. We  plan  to  gather  the  original  five 
students  and  discuss  their  reactions  to 
this  data. 

Personal  Reflections 

I was  motivated  to  work  on  this 
project  because  I want  to  influence  the 
legislature  and  policymakers  about  how 
important  adult  basic  education  is  and 
why  government  should  continue  to 
fund  and  support  its  efforts.  Although 


we  do  this  through  many  other  venues 
such  as  MCAE  (Massachusetts  Coalition 
for  Adult  Basic  Education),  NLA  (Na- 
tional Literacy  Advocacy  List),  and 
student  and  practitioner  letter-writing 
campaigns,  etc.,  I thought  another 
student  perspective  was  needed.  I 
realized  that  the  student  voice  is  most 
effective  for  this  audience,  as  we  have 
seen  in  the  past  campaign.  I also  liked 
the  idea  of  this  project  because  it  uses  an 
empirical  research  approach  to  collect 
and  analyze  the  data.  I thought  this 
would  validate  the  student  voice  in  a 
powerful  way  and  complement  the 
other  student  initiatives.  Upon  reflecting 
about  this  project,  I realize  how  much 
effort  and  thought  is  needed  to  get  a 
student  perspective.  It  doesn't  just  come 
naturally.  One  has  to  make  a conscious 
decision  to  include  students  in  all  steps 
of  the  process  and  in  meaningful  ways. 
For  example,  when  designing  the  sur- 
vey, we  could  have  made  our  own 
assumptions  about  what  questions 
should  look  like.  Instead  we  made 
a conscious  effort  to  have  students 
develop  the  questions. 

It  is  my  hope  that  our  example  of 
bringing  the  student  voice  to  the  fore- 
front will  inspire  others  to  incorporate 
the  student  perspective  in  everyday 
work,  whether  as  a teacher,  counselor, 
director,  or  administrator.  Does  the 
authentic  student  perspective  mean 
getting  students  to  write  in  a journal, 
attend  a meeting  about  program  plan- 
ning, or  fill  out  a student  survey? 

Ultimately,  answering  the  question 
entails  asking  the  students  what  they 
think  and  then  listening  to  their  ideas. 
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Feedback  101: 

Implementing  Authentic  Assessment: 
One  Program's  Perspective 


The  term  authentic  assessment  is  a 
very  broad  term  and  its  interpre- 
tation varies  from  program  to 
program.  Feedback  and  evaluation  are 
important  components  of  authentic 
assessment. 

The  International  Language  Institute 
of  MA,  Inc.  (ILI)  includes  oral  feedback 
and  written  weekly,  mid-term  and  final 
evaluations  as  part  of  its  tools  to  mea- 
sure learner  progress  and  satisfaction 
with  the  program.  ILI  has  used  this  type 
of  assessment  since  it  began  in  1984.  The 
school  has  found  that  listening  to  what 
our  learners  have  to  say  about  their 
progress  and  their  program  ensures  that 
they  have  input  into  the  curriculum,  that 
our  program  is  meeting  our  learners' 
different  needs/ goals  and  that  overall 
program  quality  is  consistently  being 
challenged  to  improve. 

Where  do  you  start  if  you're  thinking 
about  incorporating  authentic  assessment 
in  your  program?  This  article  looks  at 
how  one  school  uses  authentic  assess- 
ment and  suggests  ways  to  make  it  more 
by  accessible,  and  hopefully  more  success- 

Caroline  Gear  ful,  for  programs  and  teachers. 


One  Example:  Oral  Feedback 

When  teachers  and  programs  con- 
Intemational  sider  using  authentic  assessment,  they 

Language  Institute,  should  first  focus  on  the  theoretical 
Inc. 

Northampton,  MA  background  of  authentic  assessment.  We 
need  to  understand  why  we  are  asking 
learners  about  their  satisfaction  with  the 
class  and  their  progress  before  we  go 
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into  the  classroom  armed  with  assess- 
ment tools.  Reading  and  discussing 
articles,  talking  to  people  in  the  field 
who  are  already  working  with  authentic 
assessment,  and  analyzing  the  variety  of 
assessment  tools  available  provides  time 
for  processing  and  internalizing  the  idea 
of  authentic  assessment.  It  also  allows 
time  for  teachers  to  find /create  tools 
that  are  appropriate  to  use  with  their 
learners. 

Once  teachers  have  a philosophical 
background  in  authentic  assessment, 
they  can  start  doing  one  type  of  authen- 
tic assessment  called  oral  feedback. 
Asking  learners  about  their  progress 
and  satisfaction  with  the  program 
enables  learners  to  begin  thinking  about 
how  they  learn  and  what  works  best  for 
them.  It  also  puts  the  responsibility  on 
the  learners  to  ensure  that  the  program 
is  meeting  their  needs.  Daily  oral  feed- 
back, sometimes  after  specific  activities, 
usually  at  the  end  of  class,  is  essential 
and  has  to  be  consistent.  Oral  feedback 
also  needs  to  happen  at  the  end  of  each 
week  so  learners  can  reflect  on  what 
they  learned  in  the  past  week  and  what 
they  need  to  focus  on  in  the  coming 
weeks. 

Feeling  comfortable  about  doing  oral 
feedback  with  learners  is  not  always  an 
easy  task.  At  the  beginning  there  usually 
seems  to  be  resistance  by  both  teachers 
and  learners.  Teachers  are  uneasy  about 
being  put  into  a vulnerable  position  and 
learners  question  why  a teacher  is 
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asking  them  about  what  they  think  of 
the  class. 

A teacher  new  to  authentic  assess- 
ment had  attended  a workshop  at  ILI  on 
authentic  assessment  and  decided  to  try 
oral  feedback  in  her  class  the  next  week. 
She  said  that  the  first  three  days  she  felt 
resistance  from  the  students  and  wanted 
to  give  up.  Her  class  was  going  well, 
and  the  students  weren't  forthcoming 
with  suggestions  for  improving  the 
class.  But  she  decided  to  continue  doing 
oral  feedback  and  was  surprised  to  find 
that  as  time  went  on,  students  began  to 


talk  more  about  their  progress  and  what 


they  liked  and  disliked  about  the  class. 

Learners  need  to  feel  comfortable 
talking  about  the  class,  and  they 
shouldn't  feel  like  they  are  attacking  the 
teacher  if  they  do  not  like  a particular 
activity.  Teachers  need  to  depersonalize 
the  feedback  sessions  so  that  learners 
evaluate  the  actual  class  activities  rather 
than  the  teacher.  Once  this  routine  is 
established,  learners  will  feel  more 
comfortable  with  giving  feedback  and 
move  from  "everything  is  fine"  to 
"I  would  like  to  see....in  the  class." 

One  way  to  depersonalize  feedback 
is  to  elicit  from  the  learners  a list  of  the 
class  activities  and  record  the  list  on  the 
board.  This  gives  the  feedback  session 
structure  by  allowing  learners  to  talk 
about  what  was  covered  in  the  class 
before  assessing  the  activities.  Once 
the  list  is  created,  the  teacher  asks  the 
learners  to  rate  each  activity  according 
to  enjoyment  and  value.  A learner  may 
have  liked  an  activity,  but  doesn't  feel 
that  it  helped  them  in  the  process  of 
learning  English.  At  ILI  the  rating 
system  is  done  on  a 1-5  numerical 
system:  1 being  of  little  value  or  enjoy- 
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ment  and  5 being  of  high  value  or 
enjoyment.  A learner  can  therefore  rate 
an  activity  with  a 5 for  enjoyment  and 
a 1 for  value 

There  are  a number  of  ways  that 
this  activity  can  be  conducted.  It  also 
depends  on  the  level  of  the  class.  Begin- 
ning students  can  rate  the  activities 
based  on  a happy/unhappy  face  system, 
while  advanced  beginning  and  interme- 
diate level  learners  can  work  in  pairs 
and  then  present  their  ratings  to  another 
pair.  Pair  work  also  seems  to  help  in  the 
process  of  depersonalizing  the  activity. 

Instead  of  asking  learners  to  rate 
activities,  teachers  can  also  ask  specific 
questions  about  an  activity:  What  do 
you  think  about  this  activity?  What  did 
you  learn  from  this  activity?  Would  you 
like  to  see  this  activity  again?  This  puts 
emphasis  on  the  activity  rather  than  the 
instructor.  A teacher  new  to  feedback 
asked  her  learners:  "Did  you  enjoy  the 
song  that  I chose?"  The  way  this  ques- 
tion is  termed  does  not  allow  the  learn- 
ers to  respond  honestly  and  reflect  on 
what  they  got  out  of  the  activity 

As  teachers,  we  become  very  close  to 
the  activities  that  we  choose,  and  at 
times  we  find  it  difficult  to  step  back 
and  look  at  the  activity  without  becom- 
ing emotionally  involved.  One  teacher 
had  spent  a long  time  preparing  a 
particular  activity.  When  it  didn't  go 
well,  the  teacher  was  disappointed.  She 
felt  that  the  activity  wasn't  successful 
because  the  students  hadn't  come  to 
class  prepared.  When  asked  if  she  had 
done  feedback,  the  teacher's  response 
was  that  "she  didn't  have  time."  If  she 
had  done  feedback,  she  might  have 
found  a different  reason  as  to  why  the 
activity  didn't  go  well.  If  we  rely  on 
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As  teachers,  we 
become  very  close 
to  the  activities 
that  we  choose, 
and  at  times  we 
find  it  difficult  to 
step  back  and  look 
at  the  activity 
without  becoming 
emotionally 
involved. 
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feedback  to  help  plan  for  future  classes 
and  to  improve  our  teaching,  we  need  to 
make  time  for  feedback.  Feedback  is 
especially  important  after  activities  that 
don't  go  well. 

If  teachers  elicit  comments  from 
learners  about  the  class,  they  should  use 
the  learners'  suggestions  in  the  follow- 
ing classes.  This  validates  the  feedback 
sessions  and  makes  it  more  meaningful 
for  the  learner. 


ILI  has  found  that 
as  both  learners 
and  teachers 
become  comfort- 
able with  oral 
feedback  they  seem 
to  move  into  the 
area  of  written 
assessment  and 
evaluation  more 
easily. 


Incorporating  Authentic  Assessment 
ILI  has  found  that  if  teachers  are  to 
be  successful  with  authentic  assessment 
they  must  go  through  the  following  stages: 

• Teacher  is  exposed  to  theoretical 
background  of  authentic  assessment 
and  the  idea  of  eliciting  feedback 
from  learners 

• Teacher  practices  oral  feedback 
using  a structured  activity  of  having 
learners  list  the  activities  and  rate 
them 

• Teacher  adapts  process  according 
to  teacher  style  and  learner  needs 

• Teacher  feels  "ownership"  with  oral 
feedback  and  is  ready  to  go  to  the 
next  step  of  authentic  assessment 
All  stages  should  include  time  for 

reflection.  The  process  takes  time  and 
the  stages  may  occur  at  different  rates 
with  different  teachers.  As  teachers 
become  more  comfortable  with  the 
process,  learners  also  become  more 
comfortable  with  the  idea  of  talking 
about  their  progress  and  what  they 
want  in  the  program.  ILI  has  found  that 
as  both  learners  and  teachers  become 
comfortable  with  oral  feedback  they 
seem  to  move  into  the  area  of  written 
assessment  and  evaluation  more  easily. 
The  consistency  of  talking  about  one's 
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progress  seems  to  ease  the  transition 
to  writing  about  one's  progress. 

The  following  are  some  ideas  that 
may  make  the  process  of  adapting 
authentic  assessment  in  your  program 
a little  easier. 

Do's  and  Don'ts 

• Do  provide  readings  about  authentic 
assessment  and  provide  time  at  staff 
meetings  for  people  to  discuss  the 
pro's  and  con's  of  articles.  Every 
program  should  have  all  volumes  of 
Adventures  in  Assessment  as  it  is  an 
easy  way  to  have  a large  collection 
of  articles  and  tools  to  peruse. 

• Do  get  a copy  of  the  Tool  Kit*  and 
look  at  other  tools  that  have  been 
created  in  the  field.  Analyze  tools 
to  see  if  they  could  work  in  your 
program.  Could  you  use  this  tool 
as  is  or  would  you  have  to  adapt  it? 
Does  it  make  sense  for  you? 

• Don't  expect  that  everyone  is  going 
to  be  as  excited  about  authentic 
assessment  as  you  are  and  embrace 
it  immediately. 

• Do  contact  people  involved  in 
authentic  assessment  to  come  talk 
with  your  program. 

• Do  try  to  observe  people  using 
authentic  assessment  tools  before 
you  try  it  on  your  own. 

• Don't  start  authentic  assessment 
by  using  lengthy  assessment  tools. 
Start  off  by  doing  something  like 
oral  feedback  in  the  class. 

• Do  focus  on  depersonalizing  the 
assessment  process.  Learners  should 
be  commenting  about  the  activity 
rather  than  whether  they  like  the 
teacher. 

• Do  be  prepared  to  spend  more  time 
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than  you  thought  in  getting  authen- 
tic assessment  up  and  running  in 
your  program. 

As  ILI  enters  its  second  decade  and 
looks  back  at  all  the  written  assessment 
tools  that  it  has  used,  it  appears  that 
authentic  assessment  needs  to  be  ap- 
proached as  a journey.  Along  the  way 
we  have  created  many  assessment  tools 
and  discarded  just  as  many.  We  also 
realize  that  there  is  never  one  assess- 
ment tool  that  works  with  all  students 
and  all  teachers,  and  that  tools  to  mea- 
sure progress  take  a long  time  to  evolve. 
(See  Adventures  in  Assessment,  Spring 
1994:  "Evolution  of  an  Assessment  Tool"). 

We  know  that  the  tools  that  we  are 
using  today  will  look  different  in  a few 
years  as  the  needs  of  learners  change 
and  teachers  grow  professionally.  What 
we  have  also  come  to  realize  is  that  the 


actual  tool  to  measure  progress  is  im- 
portant, but  more  important  are  the 
teachers  that  use  the  assessment  tools. 
Teachers  who  are  new  to  authentic 
assessment  need  time  to  be  exposed  to 
and  process  the  concept  of  authentic 
assessment.  It  is  necessary  for  both 
learners  and  teachers  to  have  the  time  to 
reflect  on  what  is  going  on  in  the  class 
and  what  direction  the  class/teacher/ 
program  should  take. 

* The  Tool  Kit  is  a resource  of  assessment 
tools  put  together  by  ABE/ESL  people  in 
Massachusetts.  The  Tool  Kit  includes  intake, 
on-going  and  looking  back  assessment  tools. 
It  is  available  through  SABES. 

For  more  information  on  ILI  Authentic 
Assessment  tools,  see  AiA,  Volume  5, 
"Evaluation  of  an  Assessment  Tool,"  and 
Volume  7,  "Adventures  in  Mentoring." 
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Points  of  View 

Portfolio  Assessment  in  Rural  Maine 


by 

Marty  Duncan 


Sumner  Adult 
Education  Program 
Sumner,  Maine 


It's  early  summer  in  East  Sullivan, 
Maine.  All  of  us  who  get  through 
the  winter  here  feel  we  earn  sum- 
mer. Now  we  can  see  more  than  the 
three  winter  colors  (gray,  white,  some- 
times blue),  mud  season  is  over,  and 
the  temperature  gets  above  50  degrees 
during  the  day  — except  in  Winter 
Harbor  where,  one  of  my  students  tells 
me,  there  are  only  two  seasons:  winter 
and  the  4th  of  July. 

I'm  driving  to  the  Adult  Learning 
Center  (ALC).  As  I cross  the  singing 
bridge  that  separates  Sullivan  and  the 
neighboring  town  of  Hancock,  I see  the 
cormorants  and  gulls  gathering  on  the 
rocks,  a man  organizing  equipment  on  a 
lobster  boat,  and  the  misty  water  stretch- 
ing into  the  distance  from  both  sides  of 
the  bridge. 

The  ALC  occupies  three  of  the  four 
basement  rooms  in  the  Sorrento-Sullivan 
Recreation  Center,  home  of  the  Histori- 
cal Society,  the  Frenchman's  Bay 
Library,  and  weekly  Beano  games,  to 
name  only  a few  of  the  community 
activities.  Adult  Basic  Education,  Even 
Start  Family  Literacy,  and  the  newly- 
developing  Workforce  Literacy  compo- 
nents of  our  program  operate  at  the 
Center  as  a part  of  Sumner  Adult  Educa- 
tion. Sumner  runs  enrichment  classes, 
several  high  school  diploma  courses  and 
an  Interactive  Television  college  compo- 
nent at  Sumner  Memorial  High  School, 
several  miles  down  the  road.  One  night 
a week  during  the  fall,  winter  and 
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spring  an  adult  education  class  meets  in 
a local  elementary  school. 

I have  served  for  the  past  year  as  the 
ABE  Coordinator  at  Sumner.  Currently, 

I teach  in  the  ABE  and  family  literacy 
programs  and  coordinate  curriculum. 

In  addition,  I have  been  working  for 
the  past  year  and  a half  for  the  Horizon 
Project,  a statewide  regional  staff  devel- 
opment program  that  focuses  on  four 
areas:  assessment,  evaluation,  research 
and  reporting.  So,  my  roles  are  many 
and  varied.  In  my  own  program,  part  of 
the  region  I serve  for  Horizon,  we  have 
been  developing  portfolio  assessment. 

In  Maine,  portfolio  assessment  is  the 
recognized  assessment  in  adult  educa- 
tion as  designated  by  the  field  through 
the  state's  Quality  Indicators. 

Because  I have  been  involved  in 
portfolio  assessment  in  various  ways  — 
as  a teacher,  as  a coordinator  and  as  a 
staff  development  catalyst  — I am 
interested  in  the  various  perspectives 
that  people  involved  in  adult  education 
in  Maine  bring  to  the  subject,  especially 
now  that  we  have  been  experimenting 
with  portfolio  for  more  than  a year.  It's 
the  end  of  our  regular  cycles  at  the  pro- 
gram (because  of  summer  employment, 
most  students  participate  in  fall,  winter 
and  spring)  and  the  Horizon  Project  is  in 
its  final  phase.  I'm  in  evaluation  mode 
and  I'm  curious. 

So,  today  I intend  to  interview  eight 
people  at  Sumner  Adult  Ed.  I will  ask 
each  of  them  five  questions  about 
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portfolio  assessment.  Our  operating 
definition  is  based  on  Hannah  Fingeret's 
It  Belongs  to  Me:  A Guide  to  Portfolio 
Assessment  in  Adult  Education  Programs. 

"A  portfolio.. .can  be  a manila  folder, 
an  accordion  folder  that  extends  as  work 
is  added  to  it,  a folder  kept  on  a com- 
puter disk,  or  any  other  type  of  holder 
of  information...  Portfolios  are  collec- 
tions as  well  as  containers,"  Fingeret 
writes. 

"Portfolio  assessment  is  a process 
through  which  the  portfolio  is  created 
and  assessed.  Students  examine  a larger 
collection  of  materials  and  choose  some 
for  their  portfolios,  according  to  a set  of 
criteria.  The  portfolios  are  assessed  for 
progress,  examining  the  process  of 
learning  as  well  as  the  products.  Portfo- 
lio assessment  can  contribute  to  stu- 
dents' sense  of  control  over  their 
learning  when  they  play  central  roles  in 
their  portfolios'  creation." 

But  at  Sumner,  and  throughout  the 
state  of  Maine  as.  we  develop  the  Quality 
Indicator  process,  portfolio  assessment 
is  not  limited  to  the  assessment  of 
students'  work.  Teachers,  tutors,  coordi- 
nators, and  directors  are  all  responsible 
for  assessing  our  work  and  the  work  of 
our  programs. 

The  five  questions  I will  ask  each 
participant  are: 

1.  What  is  your  role  in  adult  education? 
(Obviously,  I know  what  their  official 
roles  are,  but  I'd  like  to  know  how 
they  define  them.) 

2.  In  which  ways  are  you  using  or 
thinking  about  using  portfolio  assess- 
ment? (Everyone  is  at  a different  stage 
of  development  with  this.  Some  on 
our  staff  have  been  recently  hired  or 


have  recently  changed  roles.) 

3.  What  do  you  like  about  it? 

4.  What  difficulties  have  you  encoun- 
tered or  do  you  anticipate  encounter- 
ing? 

5.  What  are  your  plans  for  developing 
or  continuing  to  develop  your  use  of 
portfolio  assessment? 

Thanks  to  Ann  Sargent,  ALC's 
founder  who  has  served  as  Even  Start 
Coordinator  and  will  coordinate  the 
Center  in  the  coming  year,  this  program 
was  well-grounded  in  learner-centered 
approaches  and  portfolio  assessment 
well  before  I entered  the  scene.  Ann 
participated  in  the  Quality  Indicator 
process  from  the  outset. 

As  I set  down  my  cases  and  unlock 
my  office,  I hear  them  gearing  up  in  the 
room  across  the  hall.  I'm  looking  for- 
ward to  these  interviews. 

The  noisy  activity  is  emanating  from 
the  learners  involved  in  Center's  Land- 
ing, the  math  and  critical  thinking  skills 
project  designed  by  Gail  Peterson,  the 
ABE  and  diploma  teacher,  and  a group 
of  students  who  were  "bored  with  math 
workbooks."  The  students  now  operate 
the  finances  of  several  families  and 
businesses  that  comprise  the  fictional 
town  of  Center's  Landing,  which  is 
based  on  a downeast  Maine  community. 
They  love  it. 

I'm  about  to  interrupt  them.  I want 
to  hear  from  Nancy  Austin,  a woman  in 
her  thirties  who  holds  the  distinction  of 
being  the  only  participant  in  Center's 
Landing  to  register  a dissenting  opinion 
on  one  of  the  financial  decisions.  She 
said  that  she  would  not  stand  by  and 
watch  the  group  bankrupt  these  people! 


Portfolio  assess- 
ment is  not 
limited  to  the 
assessment  of 
students'  work. 
Teachers,  tutors, 
coordinators,  and 
directors  are  all 
responsible  for 
assessing  our 
work  and  the  work 
of  our  programs. 
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One  of  the  best 
things  I've  found 
about  it  is  that  it's 
a good  way  to 
chart  your 
progress.  I see 
how  my  abilities 
got  strengthened 
academically. 


NANCY:  I'm  a student  and  I went 
back  to  the  learning  center  to  learn 
what  I didn't  learn  in  school.  I've 
used  portfolio  assessment  to  help 
me  get  ready  to  apply  for  a job  and 
to  show  what  I've  learned  as  a 
student  at  the  Center.  I like  it  be- 
cause it  shows  me  what  I have 
done — where  I started  and  where 
I was  when  I got  done.  I had  never 
done  one  before  — that  was  the 
only  difficulty  I had.  When  I do 
more  education  I can  put  more 
stuff  in  my  portfolio,  whatl'm  plan- 
ning on  learning.  For  work,  I can 
show  my  schooling,  what  back- 
ground I have. 


The  next  person  I'll  approach  is 
Craig  McKinley.  He  was  one  of  the 
group  of  learners  who  helped  establish 
Center's  Landing.  He  and  Nancy  partici- 
pated in  the  video  evaluation  of  the 
project.  Craig  has  also  participated  in 
writing  and  biology  classes  at  Sumner. 
His  11-year-old  daughter  designs  post- 
ers for  us. 


CRAIG:  Mostly  I've  been  a student 
in  adult  education  but  I trained  as 
a volunteer  tutor.  I've  worked  as  a 
volunteer  at  the  Center,  setting  up 
computer  programs.  I also  helped 
when  they  were  doing  needs  as- 
sessment — I was  kind  of  a guinea 
pig  — for  two  special  projects: 
HEAL  and  Center's  Landing. 
HEAL  [Health  Education  and  Lit- 
eracy] brought  information  about 
cancer  into  some  of  our  classes  and 
Center's  Landing  is  a math  and 
critical  thinking  project. 
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Allison  Pinkham,  another  stu- 
dent, first  explained  to  me  what  a 
basic  portfolio  is  and  passed 
around  her  own  portfolio  and  gave 
us  an  idea  about  what  we  might 
want  to  put  into  it.  (That  was  in 
English  Made  Easy.) 

One  of  the  best  things  I've  found 
about  it  is  that  it's  a good  way  to 
chart  your  progress.  I see  how  my 
abilities  got  strengthened  academi- 
cally. It's  kind  of  a self-history  that 
you  can  present  in  whatever  way 
you  choose  to  employers  or  sec- 
ondary education  people. 

I have  some  trouble  organizing 
the  information  in  a way  that  I 
think  would  make  it  easy  for  other 
people  to  look  at  it.  You  want  to 
present  yourself  in  a certain  way 
and  sometimes  it's  difficult  to  set  it 
up  like  that.  Trying  to  figure  out 
what's  pertinent  and  what's  not  is 
a problem  even  when  you  use  it 
for  yourself.  You  can  get  overly 
critical  of  your  own  things. 

To  me  a portfolio  is  kind  of  an 
ongoing  project,  never  quite  fin- 
ished. You're  always  adding  in- 
formation. It  takes  on  a life  of  its 
own  after  awhile.  It's  a verbal  pic- 
ture of  you.  I plan  to  use  it  for 
personal,  academic  and  job  pur- 
poses. 

9 

• 

While  the  Center's  Landing  group 
is  haggling  over  expenses,  I drag  Gail 
Peterson  out  of  the  classroom.  Gail  has 
taught  in  the  ABE  program  for  three 
years.  She  initiated  Center's  Landing 
at  the  learning  center,  organized  the 
materials  for  it,  piloted  it,  then  initiated 
it  in  the  group  that  meets  at  the  elemen- 
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tary  school.  Gail  also  participated  in 
Project  HEAL,  inviting  learners  in  all  of 
her  classes  to  help  us  field  test  the  kit  of 
health  education  materials.  Like  Craig, 
she  loves  to  talk,  but  she  was  leery  of 
portfolio  assessment  at  the  beginning 
and  came  to  it  slowly. 

9 

GAIL:  I am  a teacher.  I teach  writ- 
ing, GED  prep,  math,  biology  and 
chemistry. 

With  my  evening  adult  learning 
lab,  a great  deal  of  the  measure- 
ment of  what  the  students  are 
achieving  in  their  own  minds  as 
well  as  in  mine  comes  out  of  the 
comparisons  they  do  of  early  and 
later  work  that  they've  put  in  their 
portfolios.  They  are  middle-aged 
adults  so  grades  aren't  meaning- 
ful to  them.  The  portfolio  helps 
them  understand  how  they're 
learning.  I've  found  with  some  of 
the  much  younger,  still  teen-age 
learners,  just  the  physical  act  of 
putting  together  their  portfolios 
and  seeing  what  they've  done  re- 
ally helped  impress  upon  them  the 
learning  they  have  achieved. 

When  I first  started  in  adult  edu- 
cation I was  afraid  I wouldn't  be 
able  to  do  the  job.  The  portfolio  has 
helped  me  to  see  my  progress  and 
that  I did  have  the  ability.  I in- 
clude in  my  teaching  portfolio  in- 
formation about  how  courses  are 
developed,  independent  studies, 
arrangements  for  science  labora- 
tory courses,  etc.  I can  see  a huge 
change,  not  so  much  in  my  teach- 
ing style,  but  in  my  confidence  and 
depth.  That's  made  a tremendous 
difference  to  me. 

O 
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For  both  myself  and  the  students, 

I like  the  perspective  portfolio  as- 
sessment gives  us  on  what  we've 
accomplished.  I also  like  the  fact 
that  the  portfolio  is  useful  outside 
of  the  school  environment  for  em- 
ployment both  for  students  and 
teachers. 

The  time  constraints  in  the  class- 
room mean  that  portfolio  doesn't 
always  get  attention  when  it 
should.  I'm  better  now  but,  ini- 
tially, I had  a lot  of  trouble  explain- 
ing to  learners  what  this  is  and 
what  its  value  could  be.  So  a lot  of 
the  learners'  initial  contact  with 
portfolio  was  just  keeping  folders 
and  saving  everything. 

Next  semester,  I want  to  intro- 
duce it  more  concretely  earlier  on. 

I'll  be  using  my  own  portfolio  in 
class.  I want  to  do  a lot  with  the 
evening  adult  learning  lab. 

9 

• 

Now  that  Gail's  returning  to  the 
classroom.  I'll  pull  out  Peg  Walker,  one 
of  our  dedicated  volunteers.  She  also 
participates  in  Center's  Landing  and 
helped  us  with  the  video  evaluation. 
Another  straightforward,  no-nonsense 
type  of  person  (maybe  Maine  attracts 
them!),  she  has  been  known  to  rip  apart 
a document,  loaded  for  bear  when  she 
scents  jargon.  I thought  that  she  was 
more  skeptical  about  portfolio  than  she 
turned  out  to  be,  once  we  began  our 
talks  about  it. 

9 

PEG:  I volunteer  in  adult  educa- 
tion for  the  fun  of  it.  I'm  a person 
from  away  (I'm  from  outside  the 
community  originally  although  I 
live  here  full  time  now)  who  brings 
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I'm  better  now 
but,  initially,  I 
had  a lot  of  trouble 
explaining  to 
learners  what  this 
is  and  what  its 
value  could  be. 
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a different  perspective  both  in 
geogaphy  and  due  to  the  fact  that 
I was  a special  education  teacher 
rather  than  a regular  teacher.  I 
would  hope  that  this  makes  me 
able  to  be  even  more  individualiz- 
ing than  an  adult  education  pro- 
gram would  expect.  I don't  come 
in  and  say,  "This  is  what  we  do!" 

I have  no  idea  about  using  port- 
folio assessment.  I don't  know 
what  I would  do  with  it  since  I 
have  no  official  position  in  educa- 
tion. But  I think  it's  very  uniquely 
practical  for  the  person  who  is  try- 
ing to  complete  his  education  or 
re-educate  herself  to  give  the  view 
that  education  is  supposed  to  be 
practical  no  matter  what  kind  of 
orientation  it  has.  I like  the  way  it 
encourages  students  to  see  the 
things  they  have  done. 

As  long  as  it  didn't  get  too  spe- 
cific in  its  categories  I can't  imag- 
ine why  portfolio  would  give  any 
difficulties.  If  you  develop  a for- 
mat, like  a table  of  contents,  that's 
fine,  as  long  as  you  interpret  what 
goes  into  it  broadly,  with  some 
latitude. 

I plan  to  contribute  to  portfolio 
assessment  as  requested  by  either 
the  staff  or  the  students  in  the  pro- 
gram that  I volunteer  in. 

• 

Dorothy  Torrey  is  at  the  computer. 
She  is  organizing  a conference  on  work 
in  our  communities.  The  ultimate 
networker,  Dorothy  has  been  volunteer- 
ing in  and  around  our  program  for  at 
least  two  years.  This  year,  she  obtained 
a VISTA  placement  and  now  focuses 
on  developing  the  workforce  literacy 
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aspects  of  our  program.  The  newest 
member  of  the  staff,  she  enthusiastically 
avails  herself  of  staff  development 
opportunities.  Her  perspective  brings 
us  out  of  the  classroom: 

9 

DOROTHY:  I work  as  a VISTA  vol- 
unteer doing  workforce  literacy 
and  job  development.  I come  from 
a different  place  than  an  educator. 

I come  from  a business  back- 
ground. I have  not  yet  used  portfo- 
lio assessment.  However,  I clearly 
see  it  as  an  excellent  tool  for  pro- 
gram development  and  evaluation. 

I see  evaluation  as  seeing  clearly 
where  we  are  and  where  we  want 
to  go. 

I'm  truly  a novice,  but  what  I 
love  about  portfolio  assessment  is 
that  it  gives  the  space  for  visioning 
where  you  want  to  go.  Empower- 
ment — taking  learning  to  the  next 
level.  Even  though  I have  not  used 
it,  I see  it  as  valuable  in  establish- 
ing and  building  self-esteem  and 
confidence  in  the  learner.  This  is 
how  I would  want  to  use  it  — so 
that  someone  sees  clearly  who  they 
are.  Also,  I can  truly  embrace  evalu- 
ation as  a process  of  developing 
the  best  possible  program. 

A difficulty  may  arise  with  busi- 
nesses that  are  new  to  a portfolio 
approach.  I see  businesses  as  need- 
ing to  be  trained. 

I need  to  develop  a portfolio  for 
myself  so  that  I can  understand 
what  it's  like  for  a learner  to  do 
that.  So  that  I can  share  common 
experience  and  be  able  to  relate  to 
learners  as  well  as  businesses. 
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I have  to  drive  up  the  road  because, 
while  we're  on  the  subject  of  the  larger 
community.  I'd  like  to  hear  from  Roger 
Woodworth,  the  director  of  Sumner 
Adult  Education.  Roger  pre-dates  us  all. 
He  began  developing  the  adult  educa- 
tion program  in  this  community  while 
he  was  still  teaching  at  the  high  school. 
He's  been  a full-time  director  for  eight 
years.  His  relationship  with  the  commu- 
nity and  his  foresight  in  allowing  the 
development  of  a learning  center  sepa- 
rate from  the  high  school  have  contrib- 
uted enormously  to  the  growth  of  the 
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ROGER:  I am  the  director  of  a pub- 
lic school  adult  education  pro- 
gram. I have  responsibility  for  all 
facets  of  the  program:  courses, 
staffing,  publicity,  budget,  etc.  I 
work  directly  for  a school  unit  and 
am  therefore  responsible  to  the  su- 
perintendent. This  is  a full-time, 
year-around  adult  education  pro- 
gramhaving various  components: 
general  interest  courses,  adult  ba- 
sic education,  high  school  diploma 
and  GED,  a distance  education 
college  program,  and  a family  lit- 
eracy program. 

Our  adult  basic  education  and 
diploma  programs  are  moving  to- 
ward portfolio  assessment  as  the 
assessment  tool  for  student 
progress.  I see  it  as  useful  for  pro- 
gram evaluation  as  well. 

The  main  thing  I like  about  it  is 
that  the  student's  progress  and  ac- 
complishments are  not  reduced  to 
a number  or  letter.  In  the  process 
of  portfolio  assessment,  a student 
can  be  given  credit  for  a variety  of 
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activities  and  approaches.  There  is 
tremendous  flexibility  in  the  pro- 
cess. It's  student-centered  rather 
than  teacher-centered.  It  gives  a 
better  picture  of  the  whole  indi- 
vidual as  opposed  to  the  narrow 
view.  It's  a very  useful  tool  in  the 
process  of  examining  the  strengths 
and  weaknesses  of  a program.  It 
forces  you  to  take  a hard  look  at 
what  you  are  really  doing. 

Resistance  on  the  part  of  tradi- 
tional teachers  and  all  of  us,  for 
that  matter,  is  a difficulty.  It's  far 
easier  to  teach  in  the  traditional 
way  than  to  do  portfolio  assess- 
ment. It's  more  work  in  general 
and  takes  longer  — both  in  the 
classroom  and  in  the  program.  It 
takes  a certain  amount  of  faith  that 
people  will  learn  without  constant 
numerical  evaluation.  There  is  also 
some  resistance  on  the  part  of  stu- 
dents who  are  used  to  a traditional 
system. 

I'd  like  to  see  portfolio  assess- 
ment implemented  at  every  level. 

It  has  to  be  bought  into  to  be  effec- 
tive. Directors  can't  successfully 
decree  it  to  happen,  but  I,  as  a 
director,  am  in  favor  of  it  and 
would  encourage  it  at  every  op- 
portunity. 

• 

If  I hurry,  I can  catch  Jan  Johnson 
back  at  the  Center.  She'll  be  returning 
from  a home  visit  for  family  literacy, 
then  working  upstairs  in  the  early 
childhood  room  which  she  designed.  Jan 
is  the  Early  Childhood  Teacher  for  Even 
Start.  Since  she  joined  the  program,  she 
has  been  learning  about  adult  education 
according  to  the  Even  Start  model: 
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affecting  the  adult  to  affect  the  child. 
Recently,  she  showed  me  a large  scrap- 
book in  which  she  is  collecting  materials. 

9 

• 

JAN:  I see  my  role  in  adult  educa- 
tion as  being  experimental  in  the 
field  of  family  literacy.  Though  my 
background  is  in  early  childhood 
education,  I believe  that  the  long- 
range  changes  for  children  also 
need  to  include  or  come  from  the 
parents,  the  family.  So  I work  with 
the  family  as  a unit  in  our  Even 
Start  Family  Literacy  Project. 

At  this  point,  I am  saving 
samples  of  children's  work  and 
copies  of  other  materials  that  work 
with  families:  articles,  question- 
naires, documentation,  forms  like 
learning  logs,  parents'  observa- 
tions of  children.  I am  suggesting 
that  parents  save  copies  of  the 
children's  work  to  look  at  later. 

I feel  that  having  a way  to  orga- 
nize the  materials  for  future  use  in 
various  ways  is  helpful.  I feel  like 
there's  something  positive  about 
the  end  result.  You  feel  positive 
about  what  you've  done.  You  see 
progress.  I think  it  makes  our 
project  feel  more  concrete,  tangible. 

When  portfolio  assessment  is 
new,  I think  that  it  takes  a certain 
mindset  to  remember  to  save  ma- 
terials; you  need  something  to 
prompt  you.  Once  you  get  into  the 
habit,  this  shouldn'tbe  a difficulty. 
Also,  I'm  not  quite  sure  how  to 
introduce  the  concept  to  parents, 
so  at  this  point  I'm  asking  people 
to  save  things  without  being  com- 
pletely clear  about  the  purpose. 

I hope  to  develop  a plan  with  the 
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family  literacy  team  to  introduce 
family  portfolios.  I also  want,  as  a 
part  of  evaluation  this  year,  to  go 
through  what  I have  saved  and 
what  the  rest  of  the  team  has  saved 
to  examine  this  first  attempt.  I think 
that  process  will  help  for  next  year: 
to  look  at  what  I have  collected  and 
haven't  collected  will  help  me  to 
save  in  a more  organized  way  next 
year.  We're  looking  at  family  port- 
folios, a program  portfolio  and  a 
teaching  portfolio.  We  need  a sys- 
tem for  collection  and  compilation. 

9 
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Downstairs,  at  the  hub  of  the  center, 

I find  Allison  Pinkham.  Allison  has 
a special  place  for  me  because  she  was 
the  very  first  student  that  I worked  with 
when  I came  to  the  adult  learning  center. 
As  I've  told  her  many  times,  when  I first 
met  her  I didn't  think  we  would  see  eye 
to  eye.  (And  we  didn't.  I'm  five  feet  tall 
and  she's  about  5'  10"!)  Seriously, 
though,  a teacher  can  sometimes  sense 
when  the  person  who  comes  to  her  is 
going  to  be  looking  for  something 
different  from  what  that  teacher  has  to 
give.  But,  the  night  I sat  and  took  notes 
while  Allison  talked  about  all  that  she 
has  done  in  her  life  and  began  to  see 
the  ways  she  could  use  the  experience, 

I realized  that  I was  wrong.  Her  success 
speaks  for  itself.  Allison  now  serves  as 
the  secretary  and  program  assistant  for 
the  center.  She  has  accompanied  me  on 
Horizon  visits  to  other  adult  education 
programs,  and  has  shared  her  thoughts 
and  feelings. 

9 

ALLISON:  I have  many  roles:  stu- 
dent, advocate  for  students,  pro- 
gram assistant  for  family  adult 
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education.  I use  portfolio  assess- 
ment to  assess  where  I was  and 
how  far  I've  come,  mistakes  I've 
made,  not  made.  I use  it  as  a learn- 
ing tool.  I also  use  it  to  look  at  the 
program  to  see  what  could  be  im- 
proved: what  worked  and  didn't. 

I like  the  way  it  presents  things, 
shows  things  to  myself  and  other 
peopleon  whatwas  accomplished, 
things  that  I tried  that  worked  or 
didn't  work,  but  at  least  I tried. 
And  that  it  belongs  to  me,  that  I 
can  put  it  anywhere  I want  and  use 
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Documentation  is  my  biggest 
problem  with  it:  how  to  document 
things  and  finding  time,  organiz- 
ing, finding  ways  to  document 
things  like:  how  do  I document 
that  I role-modeled  for  someone? 

What  I want  to  do  next  with 
portfolio  is  talk  to  other  people 
and  programs  about  personal  or 
program  portfolios,  seeing  what 
works  for  them.  I think  it's  going 
to  change.  It's  changed  already 
over  the  last  year  or  I've  learned 
more  about  it,  or  both. 

• 

In  this  local  program  in  Maine  we 
are  continuing  to  learn  about  and  ex- 
periment with  portfolio  assessment. 

We  are  in  general  agreement  that  it  is  a 
useful  way  to  see  concretely  what  we've 
done  and  that  this  is  especially  impor- 
tant for  people  whose  achievements 
have  gone  unrecognized  or  whose  work 
has  not  been  valued.  We  also  see  that, 
with  portfolio,  a partnership  in  learning 


can  develop.  Learner  and  teacher  roles 
are  revised.  The  fact  that  learners  teach 
and  teachers  learn  becomes  clearer  and 
informs  each  new  learning  experience. 

The  way  we  view  our  roles  and 
experience  changes,  too.  Is  it  a frag- 
mented job  history  or  a collection  of 
varied  and  valuable  experience?  Is  it  a 
record  of  learning  deficits  or  a combina- 
tion of  unappreciated  strengths  and 
skills  to  be  developed?  Am  I an  instruc- 
tor, worker,  adult  student,  teacher, 
mother,  mentor,  learner,  volunteer, 
brother,  secretary?  How  do  my  roles 
fit  together  and  how  do  I understand 
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my  experience  and  communicate  it  to 
others?  Portfolio  can  begin  with  the 
pieces  of  this  puzzle  and  reveal  ways 
to  put  it  together. 

In  my  own  experience  as  a teacher. 
I've  found  that  portfolio  offers  valida- 
tion for  and  ways  to  document  the 
teaching  and  learning  approaches  that  I 
have  used  for  years.  The  most  difficult 
aspect  in  my  teaching  right  now  is 
communication  about  criteria  for  assess- 
ment. Many  (not  all)  people  hesitate  to 
assess  their  own  learning  and  some 
resist  it  fiercely.  We  are  all  trained  to 
accept  assessment  from  others  and 
through  tests,  rather  than  using  a variety 
of  assessment  approaches  in  partnership 
with  peers,  mentors  and  teachers.  The 
same  kind  of  difficulty  surfaces  in  the 
Horizon  work.  It  is  ongoing  work  to 
break  the  structures  to  allow  portfolio 
assessment  to  flourish.  My  plan,  like  the 
plans  of  the  people  I have  interviewed, 
is  to  continue  learning  — on  my  own 
and  with  others. 
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What  Counts?  A Report  to  the  Frameworks  Committee 

BRAT:  Boston  Region  Math 
Teachers  Explore  Assessment 


by 

Paula  Carranza 
Tom  Golden 
Martha  Merson 


Boston,  MA 


In  November,  1993,  several  Boston 
area  math  teachers  met  in  a pub  and 
agreed  to  use  assessment  as  a way 
to  focus  our  interest  in  and  discussions 
of  math. 

This  happy  hour  group  included 
adult  educators  from  community-based 
programs  serving  the  usual  variety  of 
speakers  of  other  languages:  low,  inter- 
mediate, and  advanced  readers,  work 
force  and  GED-bound  students,  as  well 
as  students  seeking  high  school  diplo- 
mas through  external  and  alternative 
diploma  programs. 

Taking  our  cue  from  the  statewide 
ABE  math  team,  we  agreed  to  do  math 
at  every  meeting  (hence  we  moved  to 
the  comfortable  classrooms  at  the  Com- 
munity Learning  Center  of  Cambridge). 
Over  the  course  of  a year  and  a half,  we 
found  common  ground — especially  on 
what  we  thought  was  important  to 
assess.  A stumbling  block  continued  to 
be  whether  to  focus  on  initial,  ongoing, 
or  final  assessment.  Although  the  ques- 
tion of  whether  to  develop  something 
or  write  something  as  a group  came  up 
frequently,  we  opted  to  meet  and  talk 
until  the  formation  of  study  groups  on 
the  draft  of  the  math  frameworks  pro- 
vided the  impetus  for  us  to  document 
our  thoughts. 

The  members  of  this  group  included 
Esther  Leonelli  and  Linda  Huntington, 
Juliet  Parry,  Tom  Glannon,  Janice  Forcel- 
lese,  Tom  Macdonald,  and  Kathleen 
O'Connell  and  the  three  authors. 


Assessment  is  important  to  us 
because  many  of  us  have  changed  our 
classes  according  to  the  suggestions 
originally  published  in  their  standards 
of  the  National  Council  of  Teachers  of 
Mathematics.  The  main  ideas  form  the 
basis  for  the  Massachusetts  ABE  Math 
Standards  and  for  the  frameworks  for 
teaching  mathematics  which  the  Massa- 
chusetts Department  of  Education  has 
sponsored.  All  these  documents  empha- 
size making  connections  to  real  life, 
promoting  problem  solving,  and  com- 
munication in  the  classroom. 

What  to  assess? 

These  are  the  ways  in  which  we 
want  to  assess  students'  mathematical 
abilities  and  growth.  Many  test  mea- 
surements evaluate  students  solely 
based  on  their  ability  to  come  up  with 
the  right  answer.  As  teachers,  we  would 
like  an  assessment  to  give  us  informa- 
tion on: 

Accuracy 

Logical  estimating  skills 
Application  of  accurate  computer  skills 

Communication  strategies, 
awareness  of  alternatives 
Ability  to  verbalize  at  the  beginning, 
middle,  and  end 

Strategy  repertoire 
Approach  to  the  problem 
Method  to  complete  the  problem 
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Motivation  in  other  areas 
Motivation  vs.  frustration  level 
Ability  to  start  again 

Information/  Organization 
Ability  to  assess  where  they  are 

Use  of  tools 
Use  of  strengths 

As  a group  we  brainstormed  this  list. 

In  retrospect,  Tom,  Paula,  and  Martha 
grouped  the  items  and  agreed  to  a loose 
labeling  of  them:  Skills,  Communication, 
Strategy,  Affect/ 'Attitude  (this  encom- 
passes motivation,  frustration,  and 
willingness  to  start  again),  and  Tools. 
Below  the  three  of  us  compiled  the 
group's  notes  on  the  assessment  activi- 
ties. For  each  one,  you  will  find  an 
assessment  purpose,  a description, 
process  suggestions,  timing,  and  "when 
we  did  it."  Under  assessment  purpose, 
you  will  find  references  to  the  above 
labels.  Note  that  many  activities  can 
play  more  than  one  role,  depending 
on  the  purpose  and  curriculum.  For 
example,  the  same  activity  could  assess 
Communication  and  Skills,  depending 
on  the  context. 

In  some  instances,  teachers  used  the 
assessments  in  class.  If  that  happened, 
we  have  recorded  what  we  remember 
of  our  own  and  our  students'  reactions. 
Because  our  interests  broke  down  along 
student  population  and  timing  of  the 
assessment,  we  didn't  all  have  a reason 
or  an  interest  in  trying  each  assessment 
instrument  or  activity.  Nevertheless, 
during  our  sessions,  we  were  open  to 
trying  anything  and  discussing  what- 
ever was  tried.  What  follows  are  the 
activities  we  found. 


1.  Mindbenders 

Assessment  Purpose:  Some  of  us  felt 
that  Mindbenders  and  other  logic 
puzzles  could  help  us  assess  a student's 
reasoning  process.  In  general,  they  are 
calisthenics  for  the  mind.  They  can  help 
students  identify  thinking  skills,  espe- 
cially process  of  elimination  and  deduc- 
tion from  what  is  not  known.  Use 
Mindbenders  and  similar  puzzles  to 
assess  Communication,  Strategy,  and 


Affect/ Attitude. 

Description:  The  Mindbenders  we 
used  are  from  Anita  Hamadek's  book 


— c tL  « ^ •*  lUI-ir'll 

U1  UlC  &a±i.ic  name 


A Ktt  P rifiral 

j 


Thinking  Press  (Pacific  Grove,  CA, 

1979).  The  series  has  many  levels.  The 
warm-ups  are  good,  but  we  used  some 
that  were  a little  higher  level,  A-l,  I 
believe.  The  puzzles  ask  students  to 
determine,  for  example,  which  name 
goes  with  which  profession  based  on 
certain  clues. 

Process  Suggestions:  If  students  are 
successful  at  completing  a puzzle,  ask 
them  to  write  down  the  process  they  used. 
What  were  the  steps  in  their  thinking? 

Timing:  You  could  use  logic  puzzles 
at  the  beginning  and  at  intermittent 
points  in  class  to  monitor  students' 
reasoning  processes.  However,  such 
a problem  on  an  initial  intake  might 
intimidate  students. 

When  we  did  it:  At  least  one  teacher 
said  she  didn't  like  them  and  another 
questioned  their  relevance  to  assess- 
ment. She  also  noted  that  rarely  in  life 
are  such  problems  presented.  In  our 
own  problem  solving,  we  noticed  that 
there  might  not  be  a clear,  deductive 
process,  but  rather  a risk  taken  to  pursue 
a solution  and  then  a decision  to  call  it 
solved,  check  it  or  take  a different  tack. 


ERK — —643 

MMMMilj  olnme  8:  December  1995  _ — 


ADVENTURES  IN  ASSESSMENT 


2.  Ducks  and  Lambs 

Assessment  Purpose:  A problem  to 
study  the  students'  process  of  solution. 
This  type  of  problem  would  be  a good 
on-going  assessment  tool  in  class. 
Perhaps  a form  or  set  of  steps  would 
enable  students  to  write  and  analyze 
their  own  progress.  This  would  be 
valuable  for  the  student  and  lessen  the 
burden  of  the  teacher. 

Description:  The  problem  was  stated 
and  the  group  was  asked  to  solve  the 
problem  with  a partner.  Farmer  Brown 
decided  to  take  an  inventory  of  her 
animals.  She  started  by  counting  their 
heads,  45  heads.  Next  she  counted  legs 
and  tallied  100  legs.  How  many  lambs  and 
how  many  ducks  did  Farmer  Brown  have? 

Process  Suggestions:  The  time  frame 
we  used  was  less  than  an  hour.  The  group 
agreed  that  it  was  an  excellent  activity  and 
that  it  could  be  solved  in  various  ways. 
The  different  perspectives  presented  were 
useful  insights.  There  was  not  a formal 
assessment  structure.  Everyone  was  left  to 
their  own  devices.  Make  sure  the  students 
are  encouraged  to  share  the  way  they 
came  to  their  answers. 

Timing:  The  time  we  took  for  the 
problem  was  less  than  an  hour,  but 
for  different  groups  it  could  vary. 

This  would  be  a great  on-going  or  final 
problem  to  use  if  the  curriculum  already 
had  a lot  of  open-ended  problems. 

When  We  Did  It:  Algebra  was  mainly 
used,  but  we  discussed  a variety  of  ways 
in  which  the  problem  could  be  solved.  A 
question  is  posed:  would  students  be  stim- 
ulated to  learn  algebra  if  their  method 
took  a long  time?  Another  concern  was 
for  the  teacher  evaluating  the  process. 
All  in  all  it  would  be  good  for  observing 
strategies  and  assessing  motivation. 
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3.  Computer  "Math  Blasters" 

Assessment  Purpose:  This  is  a software 
program  that  gave  an  entirely  different 
process  of  working  with  math.  The 
software  presentation  is  entertaining. 

It  builds  a student's  number  sense  and 
operation  sense  in  a visual  format.  The 
player  can  start  out  with  a trial  and  error 
process  and  advance  to  more  sophisti- 
cated techniques  to  solve  problems.  The 
assessment  is  built  in  and  is  immediate. 
Drill  is  acquired  through  an  entertaining 
formal. 

Description:  Math  Blaster  is  a com- 
puter software  program.  It  is  a well- 
designed  program  that  challenges  the 
student  level  by  level  to  increase  their 
understanding  of  numbers  and  their 
relationships.  If  the  students'  computa- 
tional skills  are  low,  a calculator  is 
provided  on  screen.  The  software  pro- 
gram provides  a wide  range  of  challenge 
and  keeps  score  for  the  users  in  chart 
form  and  gives  praise. 

Timing:  This  would  be  a good  on- 
going activity.  It  would  give  freedom 
to  students  to  work  at  their  own  level. 
The  final  assessment  would  give  an 
idea  of  the  students'  ability. 

When  We  Did  It:  Observing  the 
computer  program  for  the  first  time 
was  a treat  and  was  fascinating  to  use. 
Hands-on  use  of  the  program  gave  an 
understanding  of  the  processes  it  makes 
you  use  to  find  a solution.  It  demands 
accuracy,  makes  you  think  of  logical 
estimates,  and  practices  computer  skills. 
You  definitely  build  strategies  in  the 
approach  to  the  problem  and  solution. 
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4.  Math  Town  and  Playground 

Assessment  Purpose:  Math  Town  and 
Playground  offer  participants  the  oppor- 
tunity to  use  divergent  reasoning  and  to 
determine  and  follow  much  of  their  own 
criteria.  There  are  any  number  of  correct 
answers  so  it  simulates  many  real  life 
math  applications  where  there  is  not  a 
single  unique  correct  answer.  Partici- 
pants use  their  own  values  and  prob- 
lem-solving strategies  and  have  the 
chance  to  be  creative  in  this  open-ended 
investigation.  Use  these  problems  to 
assess  communication,  strategies,  and 
tools. 

Description:  Participants  in  pairs  or 
groups  of  three  are  given  a large  blue- 
print of  a town  with  large  open  spaces. 
They  are  told  to  designate  sites  for 
certain  features  such  as  a park  and  they 
are  to  meet  criteria  for  their  placement  in 
relation  to  other  features  in  the  town.  It 
leaves  room  for  creativity  and  an  expres- 
sion of  the  groups'  preferences  for  the 
design. 

Timing:  It  would  make  a good 
ongoing  or  final  assessment  device. 

Process  suggestions:  Assessors  should 
plan  ahead  what  they  want  to  assess. 
Participants  could  be  given  some  guid- 
ance since  it  is  very  easy  to  get  bogged 
down  in  the  details  and  miss  the  larger 
picture.  For  example,  discussions  of  the 
exact  dimensions  of  a basketball  court  or 
the  kinds  of  flowers  to  put  in  the  border 
could  distract  students  from  thinking 
about  the  design  of  the  playground 
overall. 

When  we  did  it:  Some  of  us  played 
the  role  of  participant  and  worked  in 
pairs  while  others  observed.  It  was  quite 
challenging  and  different  groups  ex- 
pressed very  different  priorities  in  their 


design.  Some  observers /assessors  were 
not  certain  what  they  were  looking  for. 

One  teacher  reported  that  "I  did  this 
in  class.  I really  wanted  more  of  them. 
The  students  found  them  most  useful. 
They  thought  it  would  be  easy  and 
didn't  expect  to  think.  I found  myself 
observing  their  thinking  and  organizing 
skills." 

In  debriefing,  we  noted  that  the 
assignment  draws  on  participants' 
background  knowledge.  The  questions 
are  not  necessarily  free  of  bias,  but  they 
do  level  the  playing  field  by  drawing 
on  many  different  kinds  of  experiences, 
e.g.,  knowing  about  basketball,  getting 
around  town,  experience  with  children 
of  different  ages. 

5.  ETS,  Quantitative/Document  Tests 

Assessment  Purpose:  These  tests 
would  have  to  be  used  in  the  proper 
environment  for  standardized  results. 
They  may  be  very  good  for  ESOL  students 
with  GEDs  to  evaluate  their  knowledge 
of  English,  using  forms  and  testing  their 
graphic  literacy  life  skills.  Two  of  the 
teachers  were  considering  this  test  as  a 
possible  replacement  or  companion  to 
the  diagnostic  test  Cambridge  currently 
uses  to  screen  entrants  to  the  External 
or  Adult  Diploma  Program. 

Descriptions:  The  ETS  Tests  or  Ap- 
plied Literacy  Skills  Document  Literacy 
and  Quantitative  Literacy  (a  set  of  two), 
would  definitely  test  skills  people  would 
need  in  the  real  world.  The  answers  are 
short,  there  is  no  multiple  choice,  and 
tsome  basic  math  is  involved. 

Process  Suggestions:  We  did  not  have 
a key  or  a table  to  score  these  tests.  No 
"norms"  were  available.  The  test  was 
too  straightforward  for  native  English 
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The  questions  are 
not  necessarily 
free  of  bias,  but 
they  do  level  the 
playing  field  by 
drawing  on  many 
different  kinds  of 
experiences,  e.g., 
knowing  about 
basketball,  getting 
around  town, 
experience  with 
children  of 
different  ages. 
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speaking  high  school  graduates  or 
GED  graduates. 

Timing:  These  tests  are  structured 
with  an  amount  of  time  for  each  section. 
A potential  innovative  way  to  use  the 
test  would  be  to  give  them  as  an  initial 
assessment  and  then  re-use  them  for 
final  assessment. 

When  We  Did  It:  Most  of  the  informa- 
tion and  questioning  was  straightfor- 
ward, but  some  students  said  they  were 
confused  about  some  of  the  map-read- 
ing questions. 

6.  Seven  Kinds  of  Smart 

Assessment  Purpose:  To  see  if  we 
could  assess  our  own  areas  of  "smart." 
We  have  all  been  frustrated  by  diagnos- 
tics that  don't  help  us  get  to  know  our 
students  better.  Use  this  inventory  to 
assess  whether  students  work  more 
efficiently  in  groups  or  alone  and  take  in 
information  visually  or  kinesthetically, 
and  to  share  information  about  students' 
strengths  with  them. 

Description:  Seven  Kinds  of  Smart  is 
a book  that  delineates  multiple  intelli- 
gences and  makes  suggestions  for  how 
to  improve  your  'smarts'  in  each  area. 
The  author  offers  a seven-part  inventory 
that  could  be  given  as  a whole  or  given  a 
part  at  a time.  The  seven  kinds  of  smart 
include:  linguistic,  musical,  logical/ 
mathematical,  spatial /visual,  bodily/ 
kinesthetic,  intrapersonal,  and  interper- 
sonal. If  you  are  dissatisfied  with  IQ 
that  measures  intelligence  along  one 
axis,  you  will  find  the  inventory  and 
its  scoring  elegant  and  empowering. 

Process  Suggestions:  One  teacher 
suggested  that  different  stations  be  set 
up  with  an  activity  or  activities  that 
relate  to  an  area.  Students  would  move 


around  the  room,  try  out  different 
activities  and  monitor  their  own 
strengths  and  attitudes.  We  agreed 
that  a disadvantage  of  this  idea  would 
be  the  teacher's  limited  ability  to  ob- 
serve everything.  In  addition,  students 
would  need  preparation  for  the  discus- 
sion to  follow.  Having  the  vocabulary  to 
describe  their  reactions  to  the  different 
stations  would  be  crucial. 

Timing:  The  inventory  could  be  used 
at  the  beginning  of  class  or  at  a middle 
point.  Its  use  to  the  classroom  teacher 
would  be  non-existent  if  done  at  the 
end. 

When  we  did  it:  Enthusiasm  was  very 
high.  We  learned  a lot  about  each  other. 
We  noted  that  as  teachers  we  would  be 
disposed  to  teach  in  the  ways  that  are 
compatible  with  our  strengths.  How 
can  we  compensate  if  our  students  are 
musically  inclined  and  we  are  not? 

We  brainstormed  ways  to  address 
each  area  within  the  context  of  math. 

A summary  follows: 

Musical:  Times  table  raps 
Spatial/visual:  graphic  representations 
Kinesthetic:  handling  manipulatives 
Linguistic:  making  up  word  problems 
Logical/mathematic:  brainteasers, 

almost  anything! 

Intrapersonal:  math  journal 
Interpersonal:  collaborative  problem 

solving,  e.g.  problems  from  SPACES. 

7.  Tactile  Math 

Assessment  Purpose:  Spending  time 
on  tactile  activities  was  inspired  in  part 
by  the  Seven  Kinds  of  Smart  discussion. 
The  original  intent,  however,  was  to 
identify  mathematical  strengths  in  a 
visually-impaired  student,  particularly 
those  not  uncovered  by  traditional 
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assessment.  Soon  it  became  obvious 
that  counting  change  was  a challenging 
activity  for  those  temporarily  deprived 
of  their  sight.  Use  this  activity  to  observe 
student's  organization,  accuracy,  and 
other  skills,  and  possible  communication 
strategies. 

Description:  The  participant  is  asked 
to  blindfold  himself  or  herself  and  to 
count  up  a set  quantity  of  change. 

Simple  as  it  sounds,  many  different 
abilities  are  required  to  accomplish  this 
task.  The  participant  has  to  keep  the 
piles  of  coins  separate  and  be  able  to 
locate  them.  Finally,  s/he  has  to  use 
mental  math  to  count  up  the  change. 

Process  Suggestions:  The  participant 
should  first  try  to  feel  the  difference 
between  coins  while  looking  at  them, 
noting  if  the  sides  of  the  coins  are  rough 


or  smooth.  The  participant  should  be 
given  three  or  four  coins  to  start  out 
with  and  work  up  to  larger  numbers  as 
s/he  becomes  proficient.  If  participants 
are  struggling,  some  coaching  is  appro- 
priate. 

Timing:  This  would  not  make  a 
good  initial  assessment.  The  participant 
should  be  fairly  comfortable  with  cre- 
ative teaching  and  ready  for  a new 
challenge. 

When  we  did  it:  One  teacher  had 
thought  about  the  activity  prior  to 
doing  it  and  did  the  task  methodically 
and  seemingly  effortlessly.  It  came  as  a 
surprise  after  having  watched  so  many 
other  participants  struggle.  This  might 
be  useful  in  finding  hidden  aptitudes 
that  might  otherwise  go  unnoticed. 
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Book  Review 

Family  Literacy  Successes 


Assessing  Success  in  Family  Literacy 
Projects:  Alternative  Approaches  to 
Assessment  and  Evaluation.  Daniel  D. 
Holt,  Editor.  Center  for  Applied  Lin- 
guistics and  Delt  Systems  Co.,  Inc.,  1994. 


Sylvie  O'Donnell 


Nantucket 
Literacy  Program 
Nantucket,  MA 


As  literacy  programs  adapt  to 

serve  the  real  life  needs  of  adult 
learners,  testing  must  be  simi- 
larly adapted  to  meet  program  needs. 
Assessing  Success  in  Family  Literacy 
Projects  offers  guidelines  for  supple- 
menting standardized  testing  with  valid 
measures  of  useful  information  relevant 
to  your  program  design. 

The  chapters  are  sequenced  to  follow 
the  order  in  which  assessment  strategies 
might  be  used  in  a start  up  program: 
selecting  members  for  the  planning 
team,  assessing  needs,  designing  the 
project,  assessing  progress,  reporting 
and  using  results.  In  following  the  steps, 
you  will  have  designed  a viable  project 
to  meet  the  needs  of  your  learners, 
teachers  and  funders.  Your  learners 
will  be  involved  in  establishing  goals; 
teachers  will  have  feedback  to  help 
direct  the  educational  process.  You  will 
have  measurable  goals  and  objectives 
for  giving  recognition  of  progress  to 
learners  and  documentation  of  achieve- 
ments for  funders. 

Chapter  1 includes  a comparison  of 
standardized  and  alternative  approaches 
to  assessment.  If  you  choose  alternative 
approaches,  the  content  of  your  tests 
will  relate  to  the  goals  and  curriculum  of 
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your  project.  Keeping  measurement  in 
mind  as  you  state  your  goals  and  objec- 
tives helps  clarify  the  purpose  and  the 
definition  of  literacy  for  your  particular 
program.  Chapter  V is  especially  helpful 
for  its  emphasis  on  the  technical  aspects 
of  collecting  and  analyzing  data,  and 
reporting  alternative  assessment  results. 
Included  are  examples  of  tools  that  have 
proven  useful  for  obtaining  information 
through  surveys,  performance  samples, 
interviews  and  observation.  Also  ad- 
dressed are  issues  of  reliability  and 
validity  in  evaluation  design. 

While  the  authors'  focus  "on  pro- 
grams serving  language-minority  adults 
and  families,  many  suggestions  could 
also  be  used  in  literacy  programs  for 
monolingual  English  speakers  (page  124)." 
The  model  approaches  used  in  developing 
the  hand-book  are  well  illustrated  in 
Chapter  IV. 

Lots  of  encouragement  and  practical 
advice  is  presented  in  detail  in  this  brief 
handbook.  The  information  provided  is 
derived  from’ the  experience  of  partici- 
pants in  family  literacy  projects,  both 
learners  and  staff.  The  book  is  clearly 
written.  The  contents  are  carefully 
outlined,  making  it  easy  to  locate  infor- 
mation you  most  want  to  review. 

The  book  should  prove  very  useful 
as  we  create  sound  alternative  ap- 
proaches to  improving  assessment, 
evaluation  and  instruction  in  our  family 
literacy  programs. 
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Book  Review 

Whole  Language  for  Adults 


Whole  Language  for  Adults;  Guide 
to  Instruction.  Chetham,  J.,  Clarke,  M., 
McKay,  D.,  Schneider,  M.,  Dunn- 
Siedow,  M.  New  Readers  Press  1994. 
Guide  to  Portfolio  Assessment: 
MacKillop,  Jane  . New  Readers  Press 
1994.  Guide  to  Initial  Assessment: 
MacKillop,  Jane.  New  Readers  Press, 
1994.  Guide  to  Administration  and 
Staff  Development;  MacKillop,  Jane. 
Nre  Readers  Press,  1995. 

Whole  Language  for  Adults 
is  a comprehensive  resource 
for  programs  interested  in 
assessing  their  practice  to  date,  reflect- 
ing on  their  needs  and  goals,  and 
implementing  authentic  teaching, 
learning  and  assessment.  A strength  of 
this  four-volume  guide  is  the  consider- 
ation given  to  issues  of  administration 
and  staff  development  in  addition  to 
instruction  and  assessment.  The  authors 
recognize  that  a whole  language  pro- 
gram requires  support  and  participation 
on  all  levels.  They  devote  a good  deal 
of  attention  to  the  role  which  adminis- 
trators must  play  in  constructing  a 
program  with  learners,  teachers,  com- 
munity members  and  funders  so  that  all 
stakeholders  are  invested  in  the  process. 

The  authors  wisely  suggest  that  a 
program  must  first  consider  what  is 
already  in  place  which  works  well  and 
which  meets  the  needs  of  the  program 
before  it  makes  sweeping  changes. 
Program  staff  are  then  encouraged  to 


reflect  on  their  goals  for  the  program 
and  how  they  can  best  be  met.  This 
process  then  drives  the  creation  of 
assessment  tools  and  instructional 
materials. 

This  is  the  other  major  strength  of 
the  guides.  Although  copy-ready  tools 
are  provided,  with  examples  of  how 
each  item  is  actually  used  in  a class  or 
program,  the  questions  which  accom- 
pany each  assessment  or  instructional 
item  are  most  valuable.  They  encourage 
practitioners  to  reflect  on  their  teaching 
and  learning  environment  and  to  adopt 
or  adapt  materials  and  practices  which 
are  best  suited  to  their  philosophies  and 
goals. 

Initial  Assessment 

The  authors  address  authentic 
assessment  in  two  separate  volumes. 
One  is  devoted  to  initial  assessment 
and  the  other  to  portfolio  assessment. 

The  guide  to  initial  assessment  is 
accompanied  by  a collection  of  reading 
selections  used  to  evaluate  oral  commu- 
nication, reading,  and  writing  proficien- 
cy. They  recommend  the  informal  read- 
ing inventory  and  auxiliary  assessment 
measures  be  used  in  place  of  standard- 
ized tests.  The  authors  also  illustrate 
how  the  information  gained  can  be 
useful  when  reporting  on  some  of  the 
indicators  of  program  quality  now 
required  for  federal  and  state  funding. 

This  guide  stresses  the  need  for 
integrating  assessment,  instruction,  and 
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program  design  and  therefore  begins 
with  very  basic  information  about 
recruiting,  planning  and  intake  strate- 
gies. When  developing  initial  assess- 
ment tasks,  educators  are  encouraged 
to  envision  " genuine  communication 
tasks"  so  that  assessment  becomes  an 
integral  component  of  curriculum 
development,  teaching,  and  learning. 
"(R)ather  than  trying  to  hide  difficulties, 
the  learner  becomes  a partner  in  explor- 
ing needs,  skills  and  goals."  This  take 
on  assessment  focuses  on  strengths  and 
recognizes  that  literacy  and  communica- 
tion involve  much  more  than  reading 
and  writing  in  workbooks. 

In  the  section  dealing  with  intake 
interviews,  the  authors  explain  the 
questions  included  in  the  intake  process. 
This  is  more  useful  than  the  actual 
interview  form,  as  it  models  the  process 
of  developing  a questionnaire  which 
meets  the  specific  needs  of  program 
staff.  The  danger  of  compiling  such  a 
comprehensive  and  detailed  volume  is 
that  programs  may  adopt  the  interview 
protocols  without  considering  their  own 
specific  needs.  For  those  who  are  just 
beginning  to  use  alternative  assessment, 
however,  this  may  provide  a solid 
foundation. 

A major  component  of  the  initial 
assessment  is  an  informal  reading 
inventory  (IRI)  comprised  of  stories 
written  by  adult  learners  and  teachers. 
New  students  are  encouraged  to  choose 
a passage  to  read  which  is  challenging 
but  not  discouraging.  The  authors  have 
ranked  the  stories  according  to  readabil- 
ity based  on  the  Fry  formula.  Although 
this  was  probably  an  effort  to  satisfy 
funders,  assigning  elementary  grade 
levels  to  stories  intended  for  adults  does 
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not  seem  very  useful,  especially  since 
the  authors  acknowledge  that  familiarity 
with  content  and  the  writing  style  of  the 
author  will  have  a strong  impact  on  the 
reader's  comprehension  of  the  story. 

I also  had  trouble  with  some  of  the 
subject  matter  of  the  stories.  The  authors 
claim  that  the  selections  were  chosen  to 
reflect  situations  in  learner's  lives;  while 
many  are  inspirational,  I found  the  stories 
about  life  on  welfare  and  in  prisons 
stereotypical.  Auxiliary  assessment 
materials  are  provided  to  "examine 
aspects  of  reading."  These  exercises  are 
incongruous  with  the  whole  language 
philosophy,  however,  for  words  are 
presented  out  of  context  in  meaningless 
lists. 

Portfolio  Assessment 

The  portfolio  assessment  planning 
guide  encourages  practitioners  to  ask 
questions  about  their  purposes  and 
goals  before  choosing  from  a variety 
of  portfolio  approaches.  Portfolios  are 
recommended  as  a recognition  of  the 
connection  between  teaching,  learning, 
and  assessing.  The  authors  argue  that 
the  use  of  portfolios  not  only  promotes 
a learner-centered  approach  to  teaching 
and  learning  but  also  offers  valuable 
documentation  of  the  work  done  in 
classes,  homes,  communities  and  work- 
places not  reflected  in  standardized 
assessments.  The  authors  also  suggest 
that  assessment  lends  itself  to  practitio- 
ner inquiry  as  teachers  reflect  on  their 
students  progress.  Again,  this  observa- 
tion recognizes  that  program  staff  need 
not  rely  upon  standardized  test  results, 
but  can  draw  on  their  own  professional 
knowledge  to  ask  and  answer  important 
questions  about  teaching  and  learning. 
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Administration  and  Staff  Development 
The  authors'  commitment  to  practi- 
tioner research  is  unclear,  however. 
While  they  recommend  that  teachers 
keep  their  own  portfolios,  this  is  not 
regarded  as  a viable  means  of  self 
evaluation  and  professional  develop- 
ment. Instead,  the  authors  emphasize 
workshops  and  training  for  staff  devel- 
opment and  teacher  evaluation.  A great 
deal  of  attention  is  placed  on  adminis- 
trators observing  teachers  while  the  use 
of  critical  reflection  on  one's  own  prac- 


tice and  dialogue  with  peers  is  not 
considered.  Also,  the  use  of  portfolios 
for  program  evaluation  and  reporting 
is  not  addressed. 

The  authors  do  recognize  that  learn- 
ers, teachers,  and  community  members, 
as  well  as  funders,  must  have  a voice 
in  the  design  and  implementation  of  a 
program  and  therefore  stress  the  impor- 
tance of  advisory  councils.  This  holistic 
approach  to  teaching,  learning,  adminis- 
tering, and  assessing  is  the  strength  of 
the  guides. 
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AN  INVITATION  TO  WRITE 

Adventures  in  Assessment  accepts  articles  regarding  any  of  the  three  compo- 
nents of  assessment:  Getting  Started,  Ongoing,  and  Looking  Back,  or  our  other 
departments:  What  Counts?,  Voices  from  the  Field,  Learning  from  Experience,  and 
Publication  Reviews. 

If  you  would  like  to  submit  an  article  to  Adventures  in  Assessment,  please 
contact  us  at: 

Adventures  in  Assessment 
Alison  Simmons,  editor 
SABES 

210  Lincoln  Street 
Boston,  MA  02111 

617-482-9485 

The  next  issue  of  Adventures  in  Assessment  will  come  out  in  Fall  1996  and 
will  highlight  writings  from  workplace  education  programs  and  low  level 
literacy  assessment  tools. 

We  welcome  your  input  and  feedback.  To  be  included  in  the  Fall  1996 
Adventures  in  Assessment,  please  contact  Alison  Simmons  at  the  above  address. 
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2 1 0 Lincoln  Street 
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Introduction 

Rethinking  Assessment:  Students 
and  Teachers  Assess  All  the  Time 


by 

Alison  Simmons 


Editor 

SABES  Central 
Resource  Center/ 
World  Education 


In  the  previous  volume  of  Adventures 
in  Assessment  I posed  the  question 
"How  will  [I]  measure  the  progress 
of  my  students  so  it  is  meaningful  to  them, 
informs  instruction  and  curriculum  and 
satisfies  funders?" 

Through  my  work  in  assessment  over 
the  year  and  working  with  authors  on  this 
volume  of  Adventures  in  Assessment,  I have 
had  a chance  to  rethink  this  question  and 
what  it  means  for  me  in  my  work  as  editor 
of  this  journal.  What  I found  confirmed  a 
belief  I have  held  since  I was  an  ESOL 
teacher.  Assessment  is  done  all  the  time 
by  students  and  teachers.  Teachers  assess 
students  initially  for  placement,  to  get  a 
sense  of  their  purpose  for  studying,  and 
for  their  skill  level. 

On  an  ongoing  basis  teachers  assess 
students  to  track  progress  and  make 
decisions  on  movement  to  other  levels  or 
to  redefine  goals  as  needed  as  students' 
lives  change  and  their  goals  are  accom- 
plished. Finally,  when  students  are  fin- 
ished with  our  services,  teachers  assess 
them  to  record  achievement  and  indicate 
where  their  skills  are  now. 

Teachers  use  a variety  of  ways  to 
accomplish  these  tasks:  observation, 
feedback  sessions,  portfolios  and  standard- 
ized tests.  For  many  teachers,  assessment  is 
integrated  into  their  instruction. 

Students  self  assess  all  the  time.  They 
seek  and  receive  feedback  from  teachers, 
community  members,  co-workers,  other 
students,  etc.  Most  of  this  feedback  is 
immediate.  Non-verbal  clues  (looks  of 
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confusion  and  impatience),  lack  of  re- 
sponse or  inaccurate  responses  by  the 
listener  are  indicators  that  there  was 
something  problematic  in  the  interaction. 

A series  of  questions  and  self  reflection 
start,  what  was  it  about  this  interaction 
that  was  problematic?  What  were  the 
barriers  preventing  the  listener  or  receiver 
from  understanding  the  interaction?  Was  it 
a bias?  What  do  I need  to  work  on  and 
what  is  out  of  my  control?  The  questioning 
process  happens  all  the  time  within  the 
students  and  helps  them  understand  the 
complex  process  of  language  learning  and 
set  goals  accordingly. 

All  this  information  is  used  to  adjust  our 
teaching,  learning  and  perceptions  of  our 
students  and  their  abilities. 

When  I asked  teachers  how  they  assess 
their  students,  they  offer  countless  anec- 
dotes and  tools  and  systems  to  move 
students  to  levels,  assess  whether  students 
can  understand  and  apply  the  materials, 
and  determine  the  immediate  needs  of  the 
students,  who  needs  other  support  ser- 
vices, etc.  Teachers  have  an  individual 
system  of  assessment  and  programs  have 
internal  systems  that  may  not  translate  to 
other  audiences. 

Teachers  know  how  to  measure 
progress  but  get  stuck  trying  to  find  a tool 
or  reporting  mechanism  to  put  their  data 
in  that  would  satisfy  all  the  different 
audiences.  The  question  goes  beyond, 

Tiow  will  I measure  the  progress  of  my 
students  so  it  is  meaningful  to  them, 
informs  instruction  and  the  curriculum 
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and  satisfies  the  hinder?' 

Rather  the  questions  should  be  'how  do 
I report  the  progress  I have  seen  to  satisfy 
different  audiences?  How  do  I report  the 
progress  my  students  have  seen?  What 
information  is  useful  to  me  in  my  program 
and  what  information  do  I need  to  satisfy 
other  audiences?  I have  seen  checklists, 
portfolios,  benchmarks,  standardized  tools 
used  as  alternative  assessment  forms, 
videos,  audio  tapes,  charts  etc.  All  docu- 
ment learning  but  are  not  transferable  to 
audiences  outside  of  the  program  or  class. 

I think  we  need  to  look  more  closely  at 

/A  r\T  i»tit rr  4"/~\  ■**/■> r\  r>  4- 
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captures  the  gains  our  students  are  mak- 
ing. We  need  to  look  at  what  teachers  are 
doing  in  the  classroom  and  students  are 
doing  and  focus  our  technical  assistance  in 
assisting  programs,  teachers  and  students 
in  documenting  and  validating  their 
assessment  tools  and  seeing  how /if  they 
connect  with  the  requirements  of  SPL 
levels  and  other  funding  mandates.  In  the 
process  we  need  to  validate  and  encourage 
teachers  to  continue  to  develop  systems 
that  inform  instruction  and  show  students 
where  they  are  making  gains  and  how  far 
they  have  come  in  achieving  their  goals. 
We  need  to  look  more  closely  into  the 
causes  of  problematic  interactions  and  not 
assume  that  it  always  lies  in  the  hands  of 
the  students. 

In  this  issue  Elizabeth  Santiago  talks 
with  six  ESOL  practitioners  about  their 
assessment  practices.  The  practitioners 
come  from  different  contexts  and  describe 
barriers  they  overcome  to  make  assess- 
ment meaningful  for  themselves  as  teach- 
ers and  for  their  learners.  They  struggle 
with  meeting  reporting  requirements  and 
have  come  up  with  tools  and  systems  that 
work  for  their  programs.  Janet  Isserlis 


talks  about  her  experience  as  a volunteer  in 
an  adult  education  program  in  Canada. 

She  looks  at  the  roles  volunteers  can  play 
in  the  assessment  process  and  how  a 
particular  assessment  tool  worked  for  her 
and  her  students  in  a conversation  class. 
Olivia  Steele,  John  Antonellis,  Jane  Shea, 
Kathe  Kirkman,  and  Debra  Tuler  discuss 
challenges  and  opportunities  in  imple- 
menting a team-based  approach  to  plan- 
ning and  evaluation  in  workplace 
education  programs  in  Massachusetts. 
Michelle  Brown  walks  us  through  the 
evolution  of  her  Native  Language  Literacy 
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mented  after  a hard  look  at  why  some 
students  were  not  progressing  in  her 
program.  She  also  introduces  us  to  the  tool 
they  developed  for  their  Native  Language 
Literacy  Program. 

In  another  article  Deborah  Mercier- 
Cuenca  talks  about  her  experience  using 
the  Native  Language  Literacy  Screening 
Device  as  a tool  for  her  program.  What  she 
found  out  through  the  assessment  process 
changed  the  way  she  approached  her  class, 
and  she,  like  Michelle  Brown,  demon- 
strates student-driven  curriculum  by 
teaching  English  Life  Skills  as  part  of  their 
Native  Language  Literacy  program. 

Tricia  Donovan  in  the  What  Counts? 
section  asks  us  to  look  a different  way  to 
assess  math  skills.  It  goes  beyond  assessing 
just  computation  to  look  at  habits  of  mind 
and  other  skills  involved  in  solving  a math 
problem.  Peg  Reidester  helps  us  look  at 
these  new  ideas  more  concretely  as  she 
takes  us  through  an  assessment  of  a math 
problem  using  a rubric. 

Finally,  Jeanne  Kearsley  reviews  an 
assessment  package  by  Cathy  Shank  that 
offers  beginning  and  experienced  teachers 
alike  a way  to  document  students'  progress. 


662 


Volume  9 • 5 


nme  9:  December  1996 


ADVENTURES  IN  ASSESSMENT 


Our  next  issue  will  be  our  tenth  and 
we  would  like  to  look  back  a few  years 
and  see  how  far  we  have  come.  There  are 
a lot  of  issues  and  questions  left  after  this 
volume  that  we  would  like  to  take  up  in 
future  volumes.  There  are  still  questions 


about  1)  the  use  and  misuse  of  the  "tests," 
2)  reporting,  and  3)  barriers  to  efficient 
assessment  processes.  We  invite  you  to 
share  your  ideas,  successes,  and  questions 
as  we  learn  from  each  other  about  our 
assessment  practices. 
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Assessment  in  the  ESOL  Experience 


"We  have  what  I call  a chat  with  very  basic  interview  questions  to  evaluate  the 
student's  oral  and  verbal  capabilities.  Now  we  have  a mandatory  test  we  have  to  give 
from  the  state.  With  the  new  test,  I sit  down  and  go  through  each  question.  Some- 
times, if  I speak  their  language,  then  I help.  But  fifty  percent  of  the  time  they  come 
with  an  interpreter.  Sometimes  I will  ask  to  call  a family  member.  Sometimes  I am 
dry.  I have  nothing. " Cecil  Demomey,  Community  Learning  Center,  Cambridge. 


raditionally,  ESOL  (English  to 
Speakers  of  Other  Languages) 
teachers  have  had  to  create  original 
assessment  tools  to  place  low-level  English 
language  learners  into  their  classrooms. 
Often  times,  standardized  tests  can  be 
inappropriate  for  ESOL  learners.  Certain 
funding  sources,  however,  require  scores 
derived  from  tests  like  the  BEST  (Basic 
English  Skills  Test)  for  funding  consider- 
ation and  some  funders  ask  for  other 
documentation  showing  how  learners 
have  progressed.  Teachers  have  had  to 
"dance"  with  what  is  required  (or  what 
they  perceive  is  required)  by  the  funders 
and  what  they  feel  will  help  them  place 
and  move  students  more  accurately. 

How  does  an  ESOL  professional  utilize 
what  is  available  to  them  when  placing 
beginning  level  students  into  their  class- 
rooms? How  about  when  they  are  tran- 
sitioning ESOL  students  into  a higher  level 
class,  working  with  a multi-level  class, 
and/ or  documenting  each  learner's 
progress  for  multiple  funding  sources? 

In  August  and  September  of  1996, 

Alison  Simmons  and  I visited  four  pro- 
grams in  Massachusetts:  Hamden  County 


House  of  Correction  in  Hamden,  Lutheran 
Social  Service  in  Springfield,  International 
Institute  in  Boston,  and  the  Community 
Learning  Center  in  Cambridge.  In  those 
diverse  settings,  we  talked  to  six  ESOL 
professionals  about  how  they  assess  low- 
level  ESOL  students  and  what  concerns 
they  have  within  that  context. 

BACKGROUND 

ESOL  services  for  adults  have  been 
gaining  national  attention  in  recent  years 
mostly  because  more  and  more  adults 
have  enrolled  in  ESOL  classes  and  waiting 
lists  for  services  are  growing.  Recent  con- 
troversies surrounding  immigrants  and 
English-only  legislation  have  put  a spot- 
light on  ESOL  learners.  Even  with  the 
current  media  coverage  of  English  lan- 
guage learners,  there  is  little  about  ESOL 
assessment  being  documented,  published, 
and  filtered  into  the  classroom.  National 
organizations  like  TESOL  (Teachers  of 
English  to  Speakers  of  Other  Languages) 
and  individual  states  and  programs 
around  the  country  have  been  researching 
and  piloting  assessment  tools,  but  it  is  not 
as  well  known  or  widespread  as  it  should 
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"The  issue  is  not 
so  much  that 
outcomes  do  not 
exist,  but  that  the 
ESL field  has 
varying  objectives 
for  instruction 
and  does  not  know 
how  to  measure 
outcomes  across 
groups." 


be.  The  Meeting  Summary  on  Current 
Research  and  Theory  on  Effective  ESL  Instruc- 
tion put  out  by  the  U.S.  Department  of 
Education  says  "The  issue  is  not  so  much 
that  outcomes  do  not  exist,  but  that  the 
ESL  field  has  varying  objectives  for  in- 
struction and  does  not  know  how  to 
measure  outcomes  across  groups" 

(page  10). 

In  a discussion  I had  with  the  editor  of 
Adventures  in  Assessment,  I expressed  the 
confusion  and  frustration  I was  feeling  in 
trying  to  assess  a young  man  that  I had 
recently  begun  tutoring.  At  our  first 
session  I wanted  to  get  a sense  where  I 
should  begin  instruction  and  what  he 
expected  from  me.  I knew  that  he  had  just 
arrived  in  this  country  and  spoke  little  if 
any  English.  I went  to  the  SABES  library  at 
the  Central  Resource  Center  and  found  a 
book  on  low-level  ESOL  assessment.  The 
book  used  pictures  and  leading  questions 
to  assess.  I decided  to  try  this  approach 
with  my  learner. 

I thought  the  first  few  pictures  might  be 
too  simple  for  him  but  I decided  to  follow 
the  format  of  the  book  and  asked  him  the 
word  for  each  object  in  English.  After  three 
objects,  he  turned  to  a friend  and  inter- 
preter and  asked  him  to  tell  me  in  English 
that  he  found  the  questions  too  easy.  What 
was  I to  do  with  the  realization  he  was 
unable  to  express  to  me  in  English  his 
concerns,  but  insisted  what  I was  asking 
him  was  too  easy?  I learned  that  my  issue 
was  one  of  many  with  which  classroom 
teachers  have  to  deal  on  a daily  basis. 

LIMITATIONS  OF  AVAILABLE  TESTS 

In  ESL  and  the  American  Dream,  authors 
Chisman,  Wrigley,  and  Ewen  state,  "There 
are  no  standardized  tests  on  the  market 
that  can  adequately  measure  the  profi- 


ciency of  the  great  diversity  of  adults  who 
need  ESL  service.  This  is  no  surprise  given 
the  complexity  of  language  skills  and  the 
range  of  abilities  that  adult  learners  dem- 
onstrate. Yet  funders  continue  to  insist  that 
standardized  assessments  be  used  to  judge 
learner  progress,  and  may  use  such  mea- 
sures of  program  effectiveness  as  well. 

This  is  true  in  spite  of  the  fact  that  the  most 
commonly  used  standardized  tests  have 
severe  limitations"  (page  58). 

What  are  some  tests  currently  in  use  by 
ESOL  teachers  and  what  are  their  limita- 
tions? There  are  both  ABE  (Adult  Basic 
Education)  and  ESOL  tests.  ABE  tests 
include:  Test  of  Adult  Basic  Education 
(TABE),  the  Adult  Basic  Learning  Exami- 
nation (ABLE),  and  the  Comprehensive 
Adult  Student  Assessment  System 
(CASAS).  These  tests  may  be  used  with 
ESOL  learners,  but  have  obvious  limita- 
tions because  they  were  not  designed  for 
that  population.  There  are  tests  specifically 
designed  for  the  ESOL  learner  which  may 
or  may  not  be  more  appropriate.  The  most 
popular  of  these  tests  include  the  BEST 
test,  the  Comprehensive  English  Language 
Skills  Assessment  (CELSA),  and  the  Test  of 
English  as  a Foreign  Language  (TOEFL). 

The  BEST  test  assesses  a learner's  basic 
survival  skills  such  as  listening  compre- 
hension, conversational  ability,  and  lit- 
eracy for  placement  purposes.  In  many 
settings,  the  BEST  test  is  required  by 
funders  as  a means  for  assessing  learners 
on  an  ongoing  basis  as  well  as  for  initial 
and  final  assessment.  In  my  discussion 
with  the  six  teachers,  each  agreed  the  BEST 
was  one  of  the  better  initial  placement 
tools  for  their  students. 

On  the  other  hand,  teachers  complained 
that  students  were  able  to  memorize  the 
items  when  the  test  is  re-administered. 


VENTURES  IN  ASSESSMENT 


Volume  9:  December  1996 


Volume  9 • I I 


therefore  making  it  difficult  to  tell  what 
knowledge  a learner  has  actually  acquired. 
Another  criticism  of  using  the  test  as  an 
ongoing  assessment  tool  is  the  idea  that 
you  have  to  teach  to  the  test.  "Sometimes 
if  you  don't  teach  to  the  test,  the  students 
aren't  going  to  pass  and  move  on  to  the 
next  level,"  said  Lola  Reid  at  Lutheran 
Social  Settlement  Services. 

The  CELSA  assesses  how  well  an 
English  learner  can  deal  with  written 
English  tests  representing  varying  levels  of 
difficulty.  "The  CELSA  is  useful  inasmuch 
as  it  provides  some  indication  on  the  lan- 
guage proficiency  level  of  ESL  students. . . 
however,  it  does  not  tell  programs  and 
funders  if  students  can  use  English  to 
achieve  their  own  purposes  in  a variety  of 
situations.  Therefore,  it  has  only  limited 
use  as  an  ESL  proficiency  assessment," 
wrote  Chisman,  Wrigley,  Ewen  in  ESL 
and  the  American  Dream. 

The  TOEFL  is  the  most  comprehensive 
of  the  available  standardized  ESOL  tests. 
TOEFL  scores  are  used  for  admission  and 
placement  into  thousands  of  colleges  and 
universities  across  the  nation.  But  the  test's 
purpose  is  narrow,  as  suggested  above, 
and  not  broad  enough  to  use  with  all  of 
our  learners. 

To  organize  my  conversations  with 
the  six  ESOL  teachers,  I categorized  the 
discussions  following  the  three  stages  of 
assessment.  The  categories  are  initial, 
ongoing,  and  final  assessment. 

INITIAL  ASSESSMENT 

"I  use  the  Oral  BEST  test.  If  they  pass,  I 
move  to  the  written  BEST.  I also  ask  lots  of 
questions,  but  nothing  personal.  I will  not 
ask  what  they  have  done  to  get  here.  I 
don't  need  to  know  that  information.  I also 
use  Scrabble  letters  to  make  words  in  their 


native  language.  If  they  use  an  alphabet 
which  is  different  than  the  English  lan- 
guage, I will  use  pictures  to  assess.  If  their 
skills  are  very  limited,  I will  try  to  interpret 
myself  or  ask  another  student  to  inter- 
pret." Eileen  Witkop,  Hamden  County  House 
of  Correction 

"I  have  an  informal  oral  interview  so  I 
can  gauge  their  level.  After  that,  there  is  an 
application  tool  with  ten  sequenced  per- 
sonal questions.  I see  how  they  respond 
and  it  helps  me  select  which  reading 
sample  to  give  them.  I have  them  read  the 
sample  silently  and  then  out  loud.  There 
are  three  or  four  comprehension  questions. 
Then  there  is  a writing  sample.  I ask  them 
to  write  about  their  family  or  something 
else  familiar.  People  usually  get  nervous 
about  spelling  so  I tell  them  up  front  that's 
not  what  I'm  looking  for  now,  but  how 
they  can  write  their  ideas.  We  also  have  an 
assessment  of  basic  math  skills,  but  appli- 
cants often  opt  out  of  this  saying  they  want 
reading  and  writing."  Leona  Breslow,  Inter- 
national Institute 

"If  I ask  their  birth  date,  for  example,  I 
will  sing,  'happy  birthday  to  you,  happy 
birthday  to  you...'  Most  people  know  the 
time  so  they  give  me  the  answer."  Cecil 
Demoumey 

Many  teachers  have  various  initial 
assessment  tools  they  use  with  their 
students,  but  there  is  not  one  tool  which 
all  the  teachers  follow.  Usually,  if  a teacher 
uses  a particular  test  to  satisfy  their 
funders,  they  also  come  up  with  some 
program-based  alternative  form  of  assess- 
ment to  use  for  themselves.  "Our  students 
come  in  at  all  levels  from  0-above.  The 
BEST  test  is  used  to  determine  the  initial 
level  of  a student.  The  numbers  generated 
from  the  BEST  test  are  the  numbers  accept- 
able by  the  funding  sources.  We  also  have 


"The  CELSA  is 
useful  inasmuch  as 
it  provides  some 
indication  on  the 
language  profi- 
ciency level  of  ESL 
students. . . how- 
ever, it  does  not 
tell  programs  and 
funders  if  students 
can  use  English  to 
achieve  their  own 
purposes  in  a 
variety  of  situa- 
tions. Therefore,  it 
has  only  limited 
use  as  an  ESL 
proficiency  assess- 
ment," 
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an  informal  assessment  of  our  own  once 
we  place  the  students  into  our  classrooms. 
We'll  list  the  activities  that  we  did  at  the 
end  of  a class.  Students  vote  on  what  they 
like  best.  This  helps  them  think  about  their 
own  learning  and  where  it  is  more  helpful 
to  them:  reading  or  writing.  Low-level  is 
more  tricky.  They  say  that  was  good.  They 
tend  not  to  be  too  critical.  If  you  ask  them 
what  they  want  to  do,  they  look  at  you 
like,  'this  is  ridiculous,  aren't  you  the 
teacher?"'  Lola  Reid,  Lutheran  Social  Services 
Settlement 

The  tests  also  raise  issues  of  inappropri- 
ateness where  the  tests  are  concerned. 

"Our  students  are  mostly  given  the  TABE 
test,  but  the  timing  is  extremely  bad.  They 
are  given  the  test,  it7 s corrected  and  put  on 
file.  We  can't  really  go  by  that  test  because 
it  is  given  at  the  worst  time.  It  is  usually 
given  right  after  someone  enters  the 
facility  and  at  that  point  they  are  usually 
in  a state  of  shock."  Eileen  Witkcrp 

Eileen's  sentiment  may,  at  first  glance, 
seem  specific  to  incarcerated  individuals. 
However,  ESOL  teachers  at  other  settings 
have  similar  concerns  when  dealing  with 
issues  of  trauma  and  shock.  Some  ESOL 
learners  are  coming  from  war-tom  coun- 
tries and  may  have  lost  family  members  or 
have  endured  other  tragedies.  Those  issues 
may  arise  in  all  ESOL  settings. 

Another  issue  in  assessing  students 
initially  and  getting  an  accurate  measure 
of  their  levels  is  the  reality  of  waiting  lists. 
Some  programs  test  students  when  they 
sign  up  for  classes,  then  place  them  on  a 
waiting  list  until  a spot  opens  up.  This 
approach  also  has  problems.  "Waiting  lists 
make  it  difficult  to  determine  their  initial 
level.  Because  they  have  been  waiting  for 
so  long,  their  English  gets  better.  Some- 
times we  have  to  make  changes.  What  we 


decided  to  do  was  network  with  other 
community  organizations  in  Cambridge 
and  help  them  set  up  English  classes. 
Waiting  lists  are  shorter  which  is  great." 
Cecil  Demoumey 

Initial  assessment  is  by  far  one  of  the 
most  difficult  areas  for  most  ESOL  teach- 
ers, but  it  has  been  one  of  the  areas  that  has 
garnered  more  documentation  (see  previ- 
ous volumes  of  Adventures  in  Assessment). 
There  are  also  many  variables  associated 
with  ongoing  assessment. 

ONGOING  ASSESSMENT 

Each  teacher  with  whom  I spoke  had 
more  ideas  for  ongoing  assessment,  but 
there  were  equally  as  many  concerns. 

"For  my  low-level  students,  I try  and 
teach  them  survival  skills.  I take  them  to 
places  like  the  supermarket  and  point  out 
fruits  and  vegetables,  show  them  how  to 
shop,  what  the  numbers  and  signs  are.  I 
show  them  how  to  look  for  an  apartment, 
learn  how  to  read  the  abbreviations.  I talk 
about  how  to  take  the  subway  and  how  to 
deal  with  getting  lost.  We  also  talk  about 
body  parts  and  medical  care.  It's  easier  to 
gauge  this  type  of  knowledge  through 
everyday  observation,  homework,  partici- 
pation in  class  and  what  the  teacher 
perceives."  Cecil  Demoumey 

The  teacher's  perception  arose  over  and 
over.  How  do  you  know  someone  is  mak- 
ing progress? 

"That  has  always  been  a question.  How 
much  learning  time  should  I spend  on 
assessment?  Informal  assessment  is  always 
in  your  head.  One  thing  that  has  been 
positive  for  me  is  after  a few  weeks  when 
they  have  a little  more  vocabulary,  I show 
them  a picture.  They  learned  the  who, 
what,  when,  etc.  prior  to  this.  I'll  ask  them, 
'who  is  this  person,  what  are  they  doing,' 
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etc.  I ask  them  to  talk  about  it,  but  not  to 
write  it.  At  the  break,  I write  the  story  they 
discussed  and  give  it  back  to  them  with 
blanks.  I would  ask  them  more  questions. 

I mention  to  them  to  look  at  all  the  English 
they  learned.  It's  very  validating."  Lola 
Reid 

"A  lot  of  it  is  work  that  they  do,  teacher 
observations,  class  work,  informal  inter- 
views, and  end-of-the-cyde  interviews. 

I will  reassess  students  who  are  having 
trouble.  We  do  see  some  students  with 
learning  issues  so  progress  can  often  be 
slow."  Leona  Bresloxv 

//!-__  i i i.  Tk-tii 
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have  just  finished  on  the  board  and  use  a 
happy  face,  a neutral  face,  and  a frowning 
face  as  the  determinant  of  each  activity. 

They  would  vote  on  which  they  preferred. 
After  a while,  I said  to  myself  why  am  I 
doing  this?  After  that,  the  students  went  to 
the  board  and  rotated  the  assessment. 

Then  I tried  it  individually  at  their  desks, 
but  this  was  for  higher  levels.  The  lower 
levels  always  did  it  on  the  board.  Then  I 
tried  an  assessment  sheet  because  I felt  the 
exerdse  was  taking  up  too  much  time  in  a 
predictable  way.  I would  give  them  this 
sheet  and  they  would  bring  it  back  at  the 
end  of  the  week.  People  didn't  like  it,  in 
fad  they  hated  it.  So  I redid  it  to  show 
their  changes.  I still  think  they  don't  like  it, 
but  they  like  this  one  better. 

It's  helpful  for  me  also  to  figure  out 
what  to  do  in  class.  Because  of  the  new 
assessment  sheet,  we  had  a conversation 
about  what  constitutes  reading,  mainly 
because  there  is  a question  that  asks  what 
have  you  read  this  week.  They  would 
come  back  and  say,  'I  didn't  read,'  but  they 
would  have  read  an  ad,  the  Yellow  Pages, 
cooking  instructions,  etc.  It's  very  easy  to 
tell,  as  a teacher,  whether  somebody  gets  it 
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or  doesn't  get  it.  It's  hard  to  say  what  I do 
exactly,  but  I just  know."  Rebecca  Schiffrin, 
Lutheran  Settlement  Services. 

Teachers  couldn't  always  pinpoint 
exactly  how  they  know  a learner  has  made 
progress.  Each  teacher  was  also  at  different 
levels  with  documentation.  Some  used 
portfolios  to  document  learner's  progress, 
some  kept  informal  notebooks,  and  many 
talked  about  goal  setting  sheets  they  had 
developed.  "I  use  portfolio  assessment  and 
goal  setting  sheets  to  tell  if  my  students 
have  progressed.  I revisit  the  goals  list 
with  the  students.  The  students  get  so 
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the  list.  Whether  it  is  the  achievement  of 
writing  their  name  in  cursive  or  watching 
a TV  show  in  English,  it  doesn't  matter 
because  they  always  feel  a sense  of  power 
when  they  check  off  something.  I also  try 
to  establish  if  there  is  a learning  disability 
present,  but  that  is  often  difficult  to  deter- 
mine. I am  building  a kit  that  is  appropri- 
ate to  ESL  learners  which  might  help  me 
see  if  there  is  a learning  disability."  Eileen 
Witkop 

"I  started  doing  some  portfolio  assess- 
ment. I would  put  job  applications  and 
other  accomplishments  in  it.  It  doesn't 
look  the  same  for  each  student  because  of 
absences.  Sometimes  I sit  with  students 
and  go  over  what  they  have  done  over 
the  last  few  months."  Barbara  Lippel  Paul, 
Lutheran  Social  Service,  Springfield 

Some  teachers  use  technology  in  their 
on-going  assessment  tools.  "Upon  comple- 
tion of  a basic  skills/ computer  class,  we 
did  on-line  classroom  assessment.  I pre- 
pared a disk  for  each  student.  On  each 
disk  was  a written  evaluation  the  student 
would  respond  to.  I would  have  leading 
questions  at  the  bottom  (see  attached  tool). 
The  students  loved  it  and  you  can  see 


Assessment  of 
learners  in  work- 
place education 
has  always  been 
challenging.  An 
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is  that  no  stan- 
dardized tests  are 
available  that 
measure  the 
learning  gains  at 
which  workplace 
education  pro- 
grams aim. 


lame  9:  December  1996 


ADVENTURES  IN  ASSESSMENT 


14  • Volume  9 


learning  styles  come  out  behind  the 
computer.  It  was  also  a learning  experience 
for  me  and  I was  able  to  modify  as  I went 
along.  I let  them  go  at  their  own  pace. 
Computer  class  was  directly  affected  by 
what  went  on  in  class.  I keep  a folder  with 
a written  evaluation.  I also  put  in  writing 
assignments  and  other  evaluation  materi- 
als." Leona  Breslow 

The  pattern  in  these  interviews  seemed 
to  be  that  there  isn't  one  way  to  do  assess- 
ment and  there  is  a level  of  intuition  on  the 
part  of  the  teacher.  Barbara  Lippel  Paul 
stated,  "You're  doing  ongoing  assessment 
all  along  in  your  class.  Everything  is  to  get 
them  out  the  door  to  the  next  level,  or  job, 
etc.  We  are  constantly  assessing."  The  next 
section  will  discuss  that  very  thing:  How 
do  you  know  when  a student  is  ready  for 
the  next  level  or  job  or  whatever? 

FINAL  ASSESSMENT 

"I  always  hear  when  a student  wants 
to  go  to  a lower  level,  but  rarely  will  a 
student  tell  me  they  want  to  go  to  a higher 
level.  When  I suspect  someone  can  go 
into  a higher  level,  I tell  them  and  they 
are  rarely  surprised.  They  sort  of  already 
know  they  are  ready  to  move  on."  Leona 
Breslow 

Knowing  when  a student  is  ready  to 
move  to  another  level,  whatever  that  level 
may  be,  takes  on  many  forms.  Some 
teachers  have  informal  interviews  with 
each  student  at  the  end  of  a class  cycle 
while  some  mentioned  checklists. 

"Some  students  like  to  stay  in  the  same 
level  because  they  know  everyone.  Some 
others  want  to  move  when  they  are  not 
ready.  Now,  we  are  trying  to  work  harder 
on  more  assessment  tools.  It's  challenging. 
We  have  a curriculum  for  each  level,  but 
we  don't  follow  it  blindly.  We  do  what's 


best  for  the  group.  We  make  sure  the 
students  master  some  grammar  because 
we  base  judgement  of  level  changes  on 
grammar.  The  problem  that  comes  up  is 
different  levels.  It's  always  a multi-level 
class.  Some  may  be  better  in  writing  than 
in  reading.  You  try  to  take  that  into  consid- 
eration when  you  are  planning  your 
lesson.  You  try  to  work  on  the  primary 
skills.  Sometimes  you  have  a student  with 
real  literacy  problems  so  you  have  to  be 
more  creative.  At  the  end  of  a cycle,  we  are 
asked  to  send  a list  to  the  counselors  of 
people  who  are  ready  to  move.  It's  tough 
though.  For  ABE  there  are  standardized 
tests.  For  ESL  there  really  isn't  anything." 
Cecil  Demoumey 

Cecil's  sentiment  sums  up  a frustration 
these  ESOL  professionals  continue  to  feel. 
What  can  we  make  of  all  this?  Well, 
Barbara,  Lola,  Rebecca,  Eileen,  Cedi,  and 
Leona  shared  with  us  their  thoughts  and 
ideas  for  assessing  their  low  and  interme- 
diate ESOL  students,  but  there  should  be 
more  documented,  researched,  and  pub- 
lished on  this  very  real  issue. 

WHAT'S  GOING  ON? 

Adventures  in  Assessment  certainly  has 
a commitment  to  exploring  all  forms  of 
altemative/authentic  assessment.  What 
other  resources  do  we  have  available  and 
what  is  being  done  to  research  this  topic? 
As  you  can  see  from  the  teacher's  work,  a 
lot  of  thought  and  hours  have  gone  into 
assessing  ESOL  learners.  Unfortunately, 
there  aren't  many  vehides  for  teachers  to 
share  what  they've  done  with  each  other. 

Or  worse  yet,  teachers  don't  feel  as  if 
they  are  doing  anything  special.  Almost  all 
the  teachers  I spoke  with  said  something 
like  "well.  I'm  sure  everyone  is  doing 
this..."  Sometimes  what  may  seem  obvious 
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or  simple  to  one  may  spark  a whole  new 
approach  to  assessment  in  someone  else. 

For  example,  this  experience  has  helped 
me  with  my  learner.  For  one,  I decided  to 
have  my  learner  guide  me.  If  he  found 
something  too  easy,  I pushed  him  to 
somehow  communicate  to  me  why  he  felt 
that  way  and  what  I could  do  to  make  my 
instruction  more  satisfying  to  his  needs.  It 
turns  out  that  he  was  an  advanced  writer 
and  reader  of  English,  but  never  really 
had  the  opportunity  to  speak  English  in  his 
native  country.  Once  this  was  established, 

I was  able  to  work  on  his  oral  abilities.  I 
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friend,  so  to  speak.  I learned  many  valu- 
able insights  from  the  interpreter  although 
he  was  not  at  every  session.  He  came  every 
now  and  again  at  the  beginning  of  our 
sessions.  We  need  to  share  more  of  what 
we  do  with  each  other. 

Adventures  in  Assessment  is  a good  place 
for  that,  as  is  the  Massachusetts  newsletter 
Bright  Ideas  or  other  state  newsletters,  but 
we  should  definitely  have  other  outlets. 
Lenore  Balliro  of  the  Adult  Literacy 
Resource  Institute  in  Boston  is  starting  an 
ESOL  listserv  on  e-mail  for  Massachusetts 
ESOL  teachers.  That  should  be  a great 


forum  for  expressing  ideas  about  assess- 
ment. Adventures  in  Assessment  will  be 
printing  another  journal  midsummer  of 
next  year;  maybe  there  could  be  more 
articles  published  on  ESOL  assessment 
in  the  upcoming  issue. 

There  also  needs  to  be  more  research 
what  learners  need  to  be  taught  and  what 
are  the  most  effective  ways  to  do  that. 
Along  with  that,  we  need  more  documen- 
tation on  how  variables  like  shock,  trauma, 
and  native  language  literacy  (among 
others)  influence  assessment  practices. 

A Center  for  Applied  Linguistics  study 
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any  language  fared  better  in  acquiring 
another  language  than  those  who  have 
no  literacy  (Meeting  Summary  on  Current 
Research  and  Theory  on  Effective  ESL  Instruc- 
tion, issued  by  the  U.S.  Department  of 
Education,  page  8).  Studies  like  this  should 
be  made  available  to  ESOL  classrooms. 

On  a national  level,  the  TESOL  conference 
is  a good  place  to  share  ideas  and  find 
workshops  on  ESOL  assessment  as  well 
as  individual  state  adult  education  confer- 
ences. In  Massachusetts,  conferences  like 
MATSOL  and  Network  '97  may  have 
ESOL  assessment  components  embedded. 
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LOOKING  AT  MANDATES 


Funders  I talked  with  do  not  mandate 
a particular  assessment  test.  Some 
teachers  have  the  perception,  however, 
that  funders  are  mandating  the  BEST  or 
TABE  for  example. 

Why?  I discovered  that  the  wording  of 
documentation  requirements  can  sometimes 
make  the  intent  confusing.  A funder,  for 
example,  may  require  a student  SPL  level 
when  they  enter  a program  and  when  they 
exit  or  move  up,  but  not  specify  a test.  One 
of  the  most  reliable  and  popular  ways  to 
arrive  at  SPL  levels  is  the  BEST.  Therefore, 
a teacher  may  translate  a requirement  of 
an  SPL  level  as  being  a mandate  for  using 
the  BEST. 

Sometimes,  too,  programs  write 
proposals  including  an  assessment  tool  and 
as  part  of  the  program  design.  What  is 
often  related  to  the  teachers  administering 
the  tests  is  not  the  funder's  requirement,  but 
what  the  program  said  it  would  do. 

There  are  funders  who  require  specific 
tests.  "The  Massachusetts  Department  of 
Education  requires  that  every  program 
funded  to  provide  ABE  services  (including 
basic  literacy  through  adult  secondary 
education  and  English  for  speakers  of  other 
languages)  incorporate  an  initial  an 


ongoing  assessment  of  each  student's  skills, 
capabilities,  and  goals.  The  Department 
expects  such  assessments  to  be  appropri- 
ate, reliable,  and  valid  measures.  The 
Department  does  not,  however,  require 
programs  to  use  any  particular  assessment 
instruments.  Programs  may  choose  from  a 
wide  variety  of  standardized,  norm  or 
criterion-referenced1  commercial  tests  and/ 
or  use  center-developed  assessments, 
including  those  referred  to  as  "alternative 
assessments",  structures,  portfolios,  etc. 

Our  goal  is  to  ensure  that  students  are 
appropriately  placed  and  that  their 
progress  is  appropriately  documented. 
Assessments  that  enable  our  students  to 
become  partners  in  managing  their  own 
learning  are  particularly  encouraged." 
(Robert  Bickerton,  ABE  State  Director,  MA) 
The  Industrial  Service  Program  in 
Massachusetts  requires  Worker  Assistance 
Centers  to  administer  the  TABE  survey  or 
the  Job  Corps  Reading  Screen  with  the 
WRAT  3.  The  Department  of  Labor 
(original  source  of  funding)  requires  that 
there  be  a reading  level  score  and  a math 
score,  not  an  equivalent,  for  each  of  the 
people  enrolled  in  their  programs.  They 
offer  a list  of  options  to  arrive  at  these 
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scores.  In  Massachusetts  the  ISP  program 
narrowed  down  the  list  to  conduct  ad- 
equate training  on  these  tools.  Some 
programs  may  use  other  tests  like  the  ABLE. 
ORI  (Office  for  Refugees  and  immigrants) 
requires  SPL  levels  for  entry  and  exit  but 
does  not  require  its  programs  to  use  the 
BEST.  If  a specific  test  is  not  required,  and 
it  is  an  SPL  level  or  a grade  level  equiva- 
lent, then  we  need  to  explore  other  valid 
and  reliable  ways  to  achieve  the  same 
outcome. 

1 Norm-referenced  tests  measure  how  well 
learners  perform  in  relation  to  a norming 
group.  Learners  are  tested  to  determine 
how  their  skills/scores  compare  with  those 
of  others  who  have  taken  the  test,  not  how 
well  they  have  mastered  certain  knowledge 
or  skills.  Criterion-referenced  tests  measure 
how  well  learners  perform  in  relation  to 
some  absolute  standard.  Learners  are 
tested  to  determine  how  well  they  mastered 
certain  knowledge  or  skills,  not  how  they 
performed  in  relation  to  a norming  group 
or  to  others  in  the  class.  ( Testing  and 
Assessment  in  Adult  Basic  Education  and 
English  as  a Second  Language  Programs) 
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Volunteer  Tutors  and  Learner 
Assessment:  What  Counts  Here? 


What  are  adult  education  pro- 
grams hoping  to  assess  when 
they  ask  their  volunteer  tutors 
to  assess  learner  progress?  This  old,  old 
question  has  been  kicked  around  in  the 
area  of  alternative  assessment  for  a very 
long  time.  Salaried  practitioners  have  been 
pondering  the  value  of  assessment  instru- 
ments for  years.  Volunteer  tutors,  how- 
ever, may  or  may  not  have  been  included 
in  the  staff  development  loop  when  ques- 
tions about  'what  counts'  are  being  asked. 
This  essay  attempts  to  address  the  fact  that 
volunteers  and  volunteer  programs  can 
benefit  as  greatly  as  any  others  from  a 
processive  approach  to  ongoing  assess- 
ment, and  suggests  some  steps  that  may 
already  be  in  place  within  many  ABE/ESL 
programs  utilizing  volunteers. 

I've  worked  in  ESL/literacy  for  the 
past  16  years,  and  since  1989  have  been 
one  of  many  people  'seriously'  considering 
assessment.  Nonetheless,  for  the  past  two 
and  a half  years,  when  asked  to  fill  in 
learner  progress  sheets  at  the  learning 
center  where  I've  volunteered  in  Van- 
couver's downtown  east  side.  I've  simply 
filled  in  the  forms  I was  given,  usually 
writing  something  general  (and  occasion- 
ally vague)  about  each  learner's  overall 
abilities.  It's  meaningless,  I figured;  some 
papers  for  the  auditor  to  count.  So  what? 

I was  just  the  volunteer  in  that  picture. 

This  past  spring  when  I was  again  given 
learner  assessment  sheets  and  told  that  the 
process  this  time  would  be  slightly  differ- 
ent, I decided  that  a shrug  and  a sigh 
weren't  helpful  responses;  volunteers  and 


learners  can  and  should  have  access  to 
information  about  and  input  into  learner 
assessment  documents.  I'd  been  given  a 
reading  from  True  Stories  in  the  News  (and 
something  else  I can  no  longer  recall)  to 
give  to  my  students.  Essentially  I was 
being  asked  to  administer  a reading 
comprehension  test  to  my  conversation 
group.  The  information  gleaned  from 
this  test  would  then  be  transferred  to 
a progress  report  (see  tool  on  page  19). 

This  time  I reconsidered.  Where  I had 
previously  written  out  something  very 
general  about  my  learners'  progress,  it 
now  occurred  to  me  that  a casually- 
written,  hollow  assessment  statement 
seemed  pointless.  If  I thought  that  the 
exercise  had  little  intrinsic  value,  (i.e.  I was 
not  convinced  that  the  people  for  whom 
the  reports  were  being  written  would 
really  read  them  closely),  I did,  however, 
see  the  possibility  of  using  the  form  as  a 
vehicle  through  which  my  learners  could 
reflect  upon  their  own  progress  in  a some- 
what more  meaningful  manner.  As  well, 
my  colleagues  at  the  center  later  pointed 
out  to  me  that  these  forms  may  in  fact  be 
used  by  center  staff  as  a form  of  in-house 
monitoring;  that  is,  the  forms  might  assist 
staff  in  shifting  learners  from  level  to  level 
as  their  abilities  increase. 

Why  discuss  this?  It's  too  easy  to  fill  in 
some  form  with  little  effort  or  substance  in 
order  to  appease  a funder.  Approximately 
the  same  amount  of  time  and  energy  could 
be  dedicated  to  a meaningful  session,  at 
the  very  least,  wherein  learners  are  asked 
to  consider  their  progress  over  a given 
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period  of  time,  and  tutors  can  share  in  the 
process  of  reflection  around  learning  gains 
and  progress. 


LEARNERS  AT  THE  CENTER 

I've  worked  with  two  core  groups  since 
I've  been  a volunteer  at  this  center.  Over 
the  course  of  two  and  a half  years,  other 
learners  have  joined  and  left,  but  this  last 
group  has  been  together  consistently  for 
more  than  a year. 

In  addition  to  the  English  as  an  Addi- 
tional Language  students  with  whom  I 
work  as  a volunteer  tutor,  the  Learning 
Centre  is  also  open  to  adult  basic  educa- 
tion (ABE)  students.  They  are  registered 
through  the  Vancouver  School  Board  and 
their  attendance  hours  are  carefully 
documented  and  counted  in  reports  which, 
in  turn  (along  with  assessment  protocols), 
are  translated  into  continued  funding 
dollars  (or  so  it  seems  — remember  I'm  a 
volunteer  and  only  part-time/ occasionally 
salaried  worker  at  this  center). 

The  learners  who  attend  the  center  are 
those  who  live  and/ or  work  in  the  down- 
town east  side  core  (the  poorest  postal 
code  in  Canada),  as  well  as  others  who 
attend  from  around  the  city.  The  issue  of 
who  attends  — who  should  or  shouldn't 
be  allowed  access  to  services  — is  a hotly- 
contested  issue,  beyond  the  scope  of  this 
essay.  This  information  is  presented  in 
order  to  give  the  reader  a sense  of  the 
context  in  which  the  volunteer  program 
has  existed. 

Throughout  the  course  of  the  year,  I 
would  periodically  ask  learners  to  discuss 
their  feelings  about  the  group  — what  was 
working,  what  wasn't,  what  needed  to  be 
changed.  Keep  in  mind  that  this  has  been  a 
conversation  group  meeting  once  a week 
for  two  hours.  Our  work  together  has 
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consisted  of  reading  and  discussing 
newspaper  articles  and  other  information 
that  learners  and/or  I bring  in.  Occasion- 
ally we'd  just  chat.  The  point  has  been  to 
build  oral/ aural  ability.  Over  time  we've 
come  to  know  one  another,  at  least  a little. 
Some  of  the  learners  phone  me  at  home  to 
ask  questions,  others  have  written  to  me 
when  I've  been  away  for  the  summer. 
We've  had  the  odd  meal  or  two  together. 
We've  spoken  periodically  about  the  group 
itself,  what  learners  like  to  do,  what  they 
want  to  learn.  Asking  them  to  consider 
their  own  progress  seemed  a legitimate 
request.  The  actual  document  used  by  the 
Centre  is  described  below. 

THE  PROGRESS  REPORT 

The  Progress  Report  form  utilized  at 
the  Centre  asks  for  the  learner's  name, 
date  of  birth,  initial  intake  date,  report 
date,  instructor's  name  and  level  at  intake. 
It  is  worth  noting  that  the  instructor  is  not 
the  same  person  as  the  tutor  (a  volunteer) 
who  is  working  with  the  learner  in  the 
tutoring  group.  As  well,  assessment  of 
level  at  intake  can  be  arbitrary,  not  stan- 
dardized in  any  way;  therefore  'progress' 
is  slippery  to  describe,  at  best.  Work  has 
been  ongoing  to  try  to  standardize  intake 
assessment.  As  far  as  I can  tell,  this  does 
not  necessarily  mean  using  standardized 
instruments  so  much  as  agreeing  upon 
levels  and  criteria  among  those  staff  who 
do  intake  work. 

The  reporting  form  given  us  for  our  ESL 
tutor  groups  appears  to  be  the  same  form 
used  to  document  progress  for  ABE 
students.  It  seeks  generalized  information 
which  may  account  for  ABE  learners' 
movement  within  levels,  but  gives  little 
room  for  real  reflection  about  ESL  learners' 
less  'visible'  progress,  such  as  the  gradual 
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shifts  in  abilities,  usage  and  confidence 
with  English.  Further,  it  is  difficult  to 
expect  a form  to  facilitate  reflection  upon 
factors  contributing  to  or  inhibiting  lan- 
guage development  — learning  about 
one's  new  environment/ culture  or  the  ups 
and  downs  which  impact  on  all  learning 
processes.  I am  critical  of  the  exercise,  not 
necessarily  of  the  learning  center.  Funders 
want  numbers,  reports,  some  sort  of 
accountability.  But  the  one  (accountability) 
does  not  necessarily  need  to  preclude  the 
other  (meaningful  information). 

After  reading  an  earlier  draft  of  this 
Daoer.  two  colleagues  at  the  Centre  told 

i i ' , u ~ 

me  that  Learning  Centre  staff  have,  in  fact, 
spoken  at  great  length  about  the  issue  of 
assessment.  It  was  suggested  that  the 
auditors  who  monitor  the  center  are  more 
interested  in  finding  completed  intake 
information  than  in  gauging  actual 


progress.  An  ongoing  communication  log, 
and  student  folders  — available  to  learners 
— serve  as  means  of  documenting  ongoing 
work.  Staff  at  the  Centre,  like  those  in 
many  others,  simply  lack  the  time  or 
institutional  support  to  refine  their  assess- 
ment work.  Instructors  are  mindful  of  the 
importance  of  meaningful  assessment,  but 
have  had  no  real  means  of  communicating 
this  to  tutors  and/or  among  one  another  in 
consistent  ways. 

The  progress  update  information  sought 
is  presented  in  the  tool  at  the  bottom  of 
this  page.  It  represents  the  actual  amount 


of  information  that's  implicitly  expected  to 


be  given.  I have  no  interest  in  attacking  the 
Centre  for  using  this  form,  but  do  want  to 
consider  ways  in  which  to  engage  in  pur- 
poseful assessment  activity  when  a funder 
asks  for  'hard  data'  and  intake  information 
and  staff  is  pressed  for  time  in  a busy 


Instructors  are 
mindful  of  the 
importance  of 
meaningful  assess- 
ment, but  have  had 
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this  to  tutors  and/ 
or  among  one 
another  in  consis- 
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center.  Funders  want  complete  student 
files  in  order  to  determine  numbers  of 
FTE  (full-time  equivalents)  and  non-FTE 
students.  My  colleagues  at  the  center  tell 
me  that  meaningful  assessment  is  impor- 
tant to  many  of  the  workers  at  the  Centre. 
Time  is  in  extremely  short  supply.  Work- 
ing day  in  and  day  out  in  a drop-in  center 
with  a shifting  learner  population  demands 
consistent  documentation  and  processes 
which  need  time  to  develop  into  useful 
assessment  protocols.  It  is  easier,  in  this 
context,  as  a volunteer  with  a very  limited 
commitment  to  an  agency,  and  with  a very 
small  group  of  learners,  for  me  to  suggest 
alternatives. 

ONE  ALTERNATIVE 

I asked  the  people  in  the  conversation 
group  to  think  about  what  they  believed 
they'd  accomplished  during  the  past  year. 

I wrote  a short  piece  to  accompany  their 
writing,  and  included  it  with  their  reports: 

Progress  Report/  1 5 May,  1 996 

I've  asked  [the  learners  in  the  group]  to 
think  and  write  about  their  own  progress 
over  the  past  year.  We  have  had  discus- 
sions about  police,  food,  restaurants,  the 
Internet,  newspaper  articles  and  book 
excerpts  that  we've  read.  We've  also  done 
some  creative  writing  and  work  on  com- 
puters. „ 

I think  that  the  core  group  has  made 
slow  and  steady  progress  and  participants 
appear  to  be  more  confident  and  comfort- 
able users  of  English.  My  learners,  who 
have  given  permission  for  their  writing  to 
be  reproduced  here  wrote  the  following 
(with  only  very  minor  editing  for  clarity). 


Sam's  report 

Since  I started  to  learn  English  one  and 
half  year  ago  I feel  my  English  had  make  a 
kind  of  progress.  I can  communicate  with  a 
lot  people,  and  talking  freely  with  people 
on  English.  I am  being  happy  on  it.  I felt 
that,  if  you  want  to  get  a express  way  on 
learning  English,  the  first  important  thing 
is  you  should  get  more  practice  in  daily 
life.  For  example  ..  I watch  the  English  TV 
often,  keep  talk  with  the  people  on  English 
often.  And  now  I am  looking  some  people 
and  make  friends  with  them.  Lastly  I have 
a lot  interest  on  talking  learning  English. 

Report  (this  learner  gave  permission  to  use 
his  writing,  hut  not  his  name) 

For  the  past  year,  I have  made  a lot  of 
progress  in  English  in  our  class  which 
Janet  taught  hard.  At  first  I have  improved 
my  hearing  and  speaking  skill  through  our 
teacher  explained  the  difficult  questions 
one  after  another  time.  Now.  I can  call  a 
phone  to  a company  to  look  for  a job  or 
ask  some  questions.  But  the  year  before 
last  year  I could  not  do  so.  Furthermore, 

I have  also  improved  my  writing  and 
reading  skills.  I can  write  a simple  business 
for  something. 

Emily's  report 

In  last  year,  I have  learned  more  English 
from  the  class.  I feel  more  comfortable  than 
before  when  use  English.  I want  to  learn 
more.  [After  Emily  had  written  the  piece 
above,  I wrote:  Emily,  can  you  think  of 
times/ places  that  are  easy  or  more  difficult 
to  speak  English,  to  which  she  responded: 
'depend  who  you  are  talk  to.' 

Hay's  Report 

Last  year,  I joined  a discussion  group  in 
[this]  Centre.  I wanted  to  improve  my 
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speaking  English.  Through  the  programs 
of  the  group,  I learned  some  new  words 
and  idioms.  I have  more  opportunity  to 
talk  in  English  too.  I feel  less  uneasy  to 
talk  with  others  in  English  now. 

These  reports  enable  us  to  reflect  on 
what  we  had  done  and  what  we  wanted  to 
do.  I imagine  that  the  learners'  notes  were 
attached  to  the  progress  report  form  and 
filed.  Additional  information  may  have 
been  added  to  the  actual  progress  report 
form,  and/ or  translated  to  fit  that  form  in 
some  way.  Who  will  read  the  reports? 
What  will  they  make  of  the  information? 
Can  or  should  th.6  learners r comments 
somehow  be  translated  into  something 
more  'standard'?  Many  learners,  particu- 
larly those  whose  first  cultures  value 
respect  for  teachers,  may  well  write  what 
they  think  the  instructor/ tutor  wants  to 
hear;  candor  is  difficult  at  best  across 
cultures  and  within  the  uneven  power 
balance  between  teachers  and  learners. 
Nonetheless,  a self-reporting  process  such 
as  this,  repeated  over  time,  may  assist 
some  learners  in  gaining  greater  insights 
into  their  own  progress.  As  trust  is  built 
between  learners  and  tutors,  more  frank 
discussions  of  what  works  and  doesn't 
work  in  the  classroom  may  ensue  as  well. 

POSSIBIUTIES/SUGGESTIONS 

As  a volunteer  tutor  at  the  Centre  in 
Vancouver,  I was  happy  spending  time 
with  learners,  sometimes  coming  in  early 
and/ or  spending  time  at  home  in  order  to 
prepare  materials.  I didn't  have  a lot  of 
extra  time,  however,  nor  do  many  volun- 
teers (and  paid  workers).  It  seems  impor- 
tant to  build  this  assessment  work  — 
especially  in  the  case  of  volunteer  workers 
— into  'regular'  hours. 

As  part  of  their  training,  volunteers/ 


new  workers  may  be  asked  to  observe 
intake  processes  (if  confidentiality  is  not 
at  issue)  and  to  discuss  with  more  experi- 
enced practitioners  the  learners'  abilities 
and  weaknesses  at  the  time  of  entry. 
Assessment  processes  could  be  discussed 
within  tutor  training  sessions  as  part  of 
the  ongoing  work  of  language/literacy 
development.  Tutors'  notes  and  logs,  along 
with  learners'  writing  samples,  should  be 
utilized  in  ways  that  help  tutors  more 
clearly  understand  learners'  progress 
without  adding  significant  blocks  of  time 
to  their  work. 

Many  programs  have  already  devised 
their  own  forms  of  assessment/ progress 
reporting.  Learners  and  practitioners 
together  could  devise  appropriate  head- 
ings/ particular  questions  for  their  sites/ 
classes  and  jointly  complete  the  reports, 
and/ or  reflect  upon  the  questions  at 
regular  intervals.  It  is  possible,  for  in- 
stance, that  my  learners'  comments  could 
be  translated  into  a grid  (see,  for  example, 
the  grid  detailed  in  Bringing  Literacy  to  Life, 
page  145).  Such  a grid  could  be  revisited 
regularly  by  learners  and  their  volunteer 
tutors;  regular  staff  could  participate  in 
this  process  periodically,  too,  in  order  to 
assist  volunteers  as  needed.  Learners  could 
be  invited  at  regular  intervals  (monthly 
or  quarterly,  for  example)  to  sit  with  staff 
and/ or  volunteers  to  review  the  items  on 
the  grid  and  to  reflect  upon  where 
progress  has  been  made,  where  work  is 
still  needed.  These  conferences  could  be 
built  into  regular  class  time,  where  every- 
one is  asked  to  write  a paragraph  reflecting 
their  feelings  about  their  own  learning  for 
the  particular  time  period.  As  learners 
work  on  their  writing,  the  facilitator(s) 
can  move  about  the  room  to  confer  with 
learners  individually. 


As  trust  is  built 
between  learners 
and  tutors,  more 
frank  discussions 
of  what  works  and 
doesn't  work  in  the 
classroom  may 
ensue  as  well. 
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While  serious  questions  have  been 
raised  about  the  use  of  volunteers  within 
literacy  work  (see  Kazemek,  1988),  many 
programs  can  and  do  utilize  volunteers 
very  effectively,  and  volunteers  make 
important  contributions  to  the  programs 
they  assist.  Why  not  enable  them  to  learn 
more  about  an  ongoing  part  of  learning 
and  teaching  so  they  can  become  yet  more 
effective? 
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Developing  a Native 
Language  Literacy  Program 


For  five  years,  Centro  Hispano de 
Chelsea  needed  a Native  Language 
Literacy  class  in  Spanish  to  meet  the 
needs  of  our  population.  When  I started  at 
Centro  Hispano,  one  program  taught  ESL 
with  a medical  emphasis  for  students  who 
tested  at  an  SPL  level  of  5-6.  This  program 
is  designed  for  1)  foreign  licensed  medical 
professionals , 2)  individuals  currently 
working  in  acute  care  hospitals,  and  3) 
individuals  pursuing  a career  in  the  health 
care  industry.  Needless  to  say,  the  students 
in  this  program  have  a much  more  estab- 
lished and  formal  educational  background 
and  were  an  excellent  match  for  this 
program. 

Initial  elation  over  the  success  of  this 
program  didn't  last  long  however.  Teach- 
ers began  getting  feedback  and  sugges- 
tions from  students  on  how  the  program 
could  better  meet  their  needs.  When  these 
suggestions  were  not  implemented, 
attrition  began  to  rise.  It  became  apparent 
that  our  education  department  was  se- 
verely lacking. 

While  we  were  meeting  the  needs  of 
our  target  population,  these  same  students 
were  asking  about  the  needs  of  their 
families.  Their  parents,  grandparents, 
brothers,  sisters,  aunts,  uncles  and  children 
were  all  in  need  of  English  and  other 
referral  services.  The  requests  for  other 
ESL  classes  were  overwhelming.  We  went 
after  funding  to  expand  our  ESL  offerings. 

We  started  to  build  our  ESL  base  with 
informal  classes  where  we  assessed  stu- 
dents via  the  BEST  (Basic  English  Skills 


Test).  Our  funding  agency  mandated  this 
tool  in  our  first  program,  so  we  considered 
it  as  good  as  any  other  tool  for  our  new 
program.  It  also  seemed  that  when  writing 
proposals  or  communicating  with  funders, 
they  understood  the  language  of  the  BEST 
SPL,  MELT  etc. 

Yet,  the  BEST  was  often  off  the  mark. 

TKo  nlarornonf  focf  nouor  coamorl 
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coincide  with  what  the  teachers  thought 
about  the  levels  of  students.  It  gave  us 
some  information  for  placement  but  not 
the  information  we  needed  to  do  it  accu- 
rately. 

Specifically,  it  did  not  account  for  all  the 
variables  in  our  ESL  population.  Students 
were  ending  up  in  classes  and  had  to  be 
moved  after  the  beginning  of  a cycle.  Still 
we  were  able  to  work  things  out.  Our  ESL 
classes  were  meeting  the  needs  of  many 
more  people.  We  had  a long  waiting  list, 
but  we  had  a program  we  felt  was  meeting 
a large  community  need. 

It  wasn't  long  however,  before  we 
noticed  a rather  peculiar  thing.  The  teach- 
ers had  brought  to  my  attention  their 
frustration  with  not  being  able  to  move  the 
classes  along  because  of  a few  students. 
While  most  of  our  students  were  making 
excellent  progress  and  were  moving  up  to 
higher  levels  of  ESL  or  out  of  the  program 
altogether,  some  were  making  little  or  no 
progress  at  all.  These  students  were 
staying  in  the  same  class  and  ESL  level  for 
upwards  of  a year  and  even  two  in  some 
instances.  As  a result,  some  beginning  ESL 
classes  were  full  and  accepted  very  few 
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see  what  those 
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new  students  from  the  waiting  list. 

We  decided  it  was  time  to  look  closely 
at  the  classes  we  were  offering.  Was  some- 
thing missing  in  our  program  that  could 
account  for  this?  We  started  by  spending  a 
whole  in-service  day  evaluating  different 
components  of  our  program  as  a whole  group. 

We  began  by  looking  at  the  numbers. 
How  many  students  had  come  in?  What 
were  their  BEST  scores  when  entering? 
What  were  their  scores  from  class  work, 
quizzes,  etc.?  and  what  were  their  BEST- 
scores  when  ending  each  cycle?  We  discov- 
ered that  most  of  the  students  who  re- 
peated two  or  more  cycles  were  in  the 
lower  level  classes,  while  the  higher  level 
classes  saw  little  to  no  repetition.  So  we 
narrowed  our  inquiry  to  lower  level  classes 
and  moved  our  focus  from  student  to 
instruction:  Was  the  teacher  appropriate 
for  the  class?  Was  the  material  appropriate 
for  the  class?  Was  the  method  of  teaching 
appropriate  for  the  class?  The  majority  of 
the  class  was  doing  well  and  showed 
mobility  into  the  higher  level  classes.  All 
teacher  evaluations  written  by  students 
had  come  out  very  positive,  the  class 
curriculum  was  suitable,  and  the  course 
work  all  seemed  to  be  in  order. 

One  of  the  teachers  suggested  we  ask 
all  the  students  in  the  lower  level  classes  to 
write  a short  paragraph  in  Spanish  (their 
native  language),  perhaps  about  their  early 
educational  experiences.  We  expected 
some  issues  with  their  native  language  but 
we  wanted  to  see  what  those  issues  were. 
Was  it  a literacy  problem?  Was  it  an  issue 
of  a bad  educational  experience?  Was  it  a 
result  of  a learning  style  difference?  After 
the  students  provided  us  with  a writing 
sample,  the  results  were  so  clear  and 
relevant  that  they  gave  us  two  major 
pieces  to  the  puzzle. 


The  first  was  that  the  lesser-developed 
paragraphs  with  the  most  mistakes  all 
belonged  to  those  students  who  had 
remained  in  the  lower  level  class  for  more 
than  two  cycles.  These  mistakes  ranged 
from  spelling  errors,  grammatical  mis- 
takes, poor  sentence  structure,  to  mal- 
formed letters,  such  as  backwards  p's 
etc. . . .also  the  actual  writing  on  the  page 
was  not  smooth  flowing  but  looked  as  if 
a lot  of  effort  was  put  into  writing  each 
word. 

The  second  was  that  all  the  students 
were  over  55  years  of  age.  None  of  these 
students  had  education  in  their  own 
country  beyond  the  third  grade  (even  that 
was  considered  advanced);  some  had 
absolutely  no  formal  education  in  either 
reading  or  writing.  All  considered  them- 
selves "campesinos,"  persons  who  grew 
up  on  a farm.  Their  family  priorities  were 
to  work  on  their  farms  starting  at  a very 
young  age  so  they  had  few  opportunities 
to  further  their  education  in  their  native 
language. 

We  needed  a class  to  fit  these  specific 
Native  Language  Literacy  (NLL)  needs. 
We  believe  that  people  have  better  success 
at  learning  literacy  of  a second  language 
if  they  have  literacy  skills  in  their  native 
language.  This  belief  led  us  to  pursue 
funding  for  a Native  Language  Literacy 
Class.  Centro  Hispano  de  Chelsea  then 
joined  forces  with  Bunker  Hill  Community 
College  and  other  agencies  in  Chelsea: 
Chelsea  City  Hall,  Chelsea  Community 
Volunteer  Center,  Cambodian  Community 
of  Massachusetts,  Refugee  Services, 
Consilio  Hispano  de  Cambridge,  and  the 
Chelsea  Community  to  respond  to  a 
request  for  proposals  through  the  Massa- 
chusetts Department  of  Education. 
Through  this  consortium  we  are  able  to 
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provide  ESOL,  GED  and  pre-GED  in 
Spanish  and  English  and  two  NLL  pro- 
grams in  Spanish  and  Khmer. 

We  could  now  finally  start  our  program, 
but  the  next  stumbling  block  was  finding 
or  developing  the  proper  assessment  tool. 
As  previously  stated,  we  had  used  the 
BEST  for  our  other  programs  as  well  as 
supplemental  materials.  This  test  could 
not  address  our  need  to  measure  Native 
Language  Literacy.  We  had  to  look  else- 
where for  a tool. 

We  looked  at  various  alternatives, 
including  resource  people  like  Yvonne 
Lalyre  at  the  Mass.  Department  of  Educa- 
tion and  Seidy  Rodriguez-Ham,  our 
literacy  teacher.  Seidy  drafted  our  own  in- 
house  Native  Language  Literacy  Assess- 
ment Tool  (see  Tool,  next  page).  This  tool 
began  to  assess  students'  knowledge  of 
the  alphabet,  sentence  structure,  and  basic 
math  applications.  With  invaluable  assis- 
tance from  Diane  Paxton,  the  Elderly 
Program  Life  Skills  instructor  at  Centro 
Hispano,  we  were  able  to  get  an  even  more 
advanced  assessment  and  form  a more 
comprehensive  curriculum  for  the  NLL 
program.  Seemingly  all  segments  were  in 
place,  but  there  was  one  last  - and  prob- 
ably the  largest  problem  left. 

We  had  identified  the  students  who 
needed  the  classes  and  had  found  an 
instructor.  We  had  then  developed  the 
necessary  assessment  tools  as  well  as  the 
curriculum.  Yet  the  students  were  ex- 
tremely dissatisfied , and  many  of  them 
refused  to  come  to  the  class.  Many  of  them 
denied  their  need  for  classes  in  native 
language  literacy.  Others  admitted  they 
needed  to  learn  to  read  and  write  in  their 
own  language,  but  wanted  and  needed  to 
learn  English.  There  was  also  the  issue  of 
being  viewed  as  an  "alphabeta(o)"  and  the 


stigma  attached  to  not  being  able  to  read 
and  write.  This  was  hard  for  some  students 
to  overcome.  It  was  also  hard  for  us  to 
convince  students  that  this  was  the  best 
path  to  learning  English.  Some  students 
resisted  and  wanted  to  just  learn  English. 
After  listening  to  their  concerns,  we  then 
developed  what  we  hoped  would  be  our 
last  solution  to  the  issues  surrounding  our 
Native  Language  Literacy  Program. 

We  offered  the  students  a supplemen- 
tary class  which  taught  ESL  life  skills 
(shopping,  emergency  procedures,  going 
to  the  doctor. . .).  They  could  take  the 
literacy  class  Monday  through  Wednesday 
and  take  the  Life  Skills  class  Thursday  and 
Friday.  This  would  assist  them  with  their 
literacy  skills,  while  at  the  same  time  teach 
them  survival  English.  This  option  was 
extremely  popular  and  even  helped  us 
solve  other  problems  we  were  having  in 
our  ESL  classes  mainly  dealing  with  issues 
of  stereotypes  and  prejudices  from  stu- 
dents coming  from  different  parts  of  Latin 
America. 

Students  coming  from  some  Latin 
American  countries  are  very  strong  in  their 
English  reading  and  writing  skills,  seem- 
ingly due  to  learning  English  in  school, 
some  as  early  as  first  grade.  This  English 
instruction  was  generally  taught  in  Span- 
ish and  emphasized  reading  and  writing, 
yet  did  little  to  develop  their  oral/ aural 
skills. 

Individuals  from  other  Latin  American 
countries,  however,  have  been  in  this 
country  for  a long  time  and  have  relied 
much  more  on  the  oral/ aural  skills  to 
survive  and  work  and  have  had  few  or 
no  opportunities  to  develop  their  reading 
and  writing  skills. 

Our  Multi  Level  ESL  Life  Skills  class 
served  a dual  purpose  in  the  education , 
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employment  and  training  department.  Not 
only  did  it  see  success  with  our  diverse 
students  from  Latin  America,  but  also 
helped  up  to  implement  and  see  success  in 
our  Native  Language  Literacy  Program. 

Our  two  classes  brought  together  students 
from  different  countries  in  Latin  America 

ET 


for  a common  purpose.  Through  these 
classes  we  were  able  to  facilitate  conversa- 
tions and  learnings  that  helped  people  see 
the  similarities  and  differences  in  their 
languages,  phrases  and  terminology  and 
as  a result  gave  students  a more  accurate 
picture  of  the  different  cultures. 
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TOOL:  CONTESTE  LO  SIGUIENTE 

Nombre Fecha 

1 . Escriba  las  Vocales. 

2.  Escriba  las  letras  del  abcedario. 

3.  Forme  palabras  con  las  siguientes  letras. 

M 

S 

P 

L 

N 

4.  Forme  oraciones  con  las  siguientes  palabras 
mama 

luna 

nene 

casa 

papa 
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Complete  el  siguiente  parrafo 

5.  Escriba  en  la  raya  la  letre  que  falta 

Tengo  ami os  son  Luis  y Mario aseamos  juntos 

po el  campo  jugamos uando  salimos  de  la 

escuela  es  muy onito. 

6.  Escriba  en  cada  raya  los  numeros  que  estan  antes  y despues  de: 

4 

10 

20 

37 

49 
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Planning  and  Evaluation  Teams: 

A Model  for  Workplace  Education 


Evaluating  the  quality  and  effectiveness  of  workplace  education  programs  creates 
both  opportunities  and  challenges.  To  meet  the  challenges,  the  Massachusetts  Work- 
place Education  Initiative  promotes  a team-based  approach  to  program  planning, 
implementation,  evaluation,  and  institutionalization.  The  approach  uses  a "Plan- 
ning and  Evaluation  Team"  model.  Since  its  inception  in  1991,  the  model  has  been 
implemented  in  over  50  workplace  education  programs.  This  article  describes  the 
issues  that  have  surfaced  and  includes  a response  to  them  from  four  practitioners. 


by 

Olivia  Steele, 
Deborah  Toler, 
Jane  Shea, 
John  Antonellis, 
Kathe  Kirkman 


Massachusetts  Workplace 
Literacy  Program 


Evaluating  the  effectiveness  of 
workplace  education  programs 
creates  exciting  opportunities.  If 
done  well,  evaluation  results  can  demon- 
strate concrete  and  tangible  benefits  of  the 
program  to  the  students,  the  employer,  the 
union,  and  in  some  cases  the  community. 
Evaluation  results  — including,  but  not 
limited  to,  assessment  results  — can  show 
what  was  learned,  how  that  learning 
transferred  to,  and  got  used  at,  the  work- 
place and  in  the  community,  and  to  what 
extent  participation  in  the  program  im- 
pacts on  the  performance  of  the  business. 
Evaluation  results  can  also  show  to  what 
extent  the  improvement  of  basic  or  lan- 
guage skills  contributes  to  increasing  the 
quality  of  goods  and  services  and  improv- 
ing production  or  service  delivery  pro- 
cesses. It  can  inform  program 
improvement,  can  promote  institutional- 
ization of  the  program,  and  can  help 
determine  whether  to  continue  funding. 

Evaluating  the  effectiveness  of  work- 
place education  programs  also  creates 
challenges.  Identifying  gains  in  student 


learning,  describing  the  transfer  of  that 
learning,  documenting  the  impact  of  the 
program  on  the  workplace,  and  demon- 
strating the  program's  overall  quality  and 
effectiveness  have  been  daunting  tasks  for 
many  practitioners.  Several  factors  make 
evaluation  challenging.  One  important 
factor  is  that  workplace  education  pro- 
grams are  not  yet  part  of  the  workplace's 
infrastructure  and  culture  when  they  are 
initiated.  Establishing  this  infrastructure 
and  culture,  as  well  as  building  systems  to 
take  care  of  planning,  implementation,  and 
evaluation  functions  need  to  take  place 
simultaneously  and  require  a considerable 
amount  of  resources  and  energy. 

Another  factor  is  the  involvement  of 
many  different  players,  including  students, 
teachers,  administrators,  employers,  and 
union  representatives,  all  of  whom  bring 
their  own  ideas  and  expectations  regard- 
ing what  constitutes  success.  Striking  an 
appropriate  balance  between  these  differ- 
ent expectations  is  not  an  easy  task. 

The  varying  degree  of  involvement  in 
the  program  by  those  who  have  a vested 
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interest  in  it  or  have  decisionmaking 
authority  over  it,  is  yet  another  important 
challenge  to  evaluating  a program.  When 
key  decision  makers  such  as  Chief  Execu- 
tive Officers  are  involved  directly,  plan- 
ning, implementation,  evaluation,  and 
institutionalization  decisions  appear  to 
be  facilitated  more  easily.  High  levels  of 
managerial  involvement,  however,  have  in 
some  eases  created  an  imbalance  of  power. 
When  key  decisionmakers  are  not  in- 
volved directly  in  the  planning  and  evalu- 
ation of  a program,  those  who  are  need  to 
invest  considerable  time  in  keeping 
decisionmakers  fully  apprised.  Tf  not,  the 
program  may  end  up  being  decided  upon 
from  a distance  by  someone  based  only 
on  his/her  expectations  and  perceptions 
as  to  how  responsive  the  program  is.  This 
perception  may  not  be  an  accurate  reflec- 
tion of  the  program's  performance. 

Evaluating  workplace  education  pro- 
grams is  most  definitely  challenging 
because  of  the  expertise  required  to 
develop  and  implement  an  evaluation 
system.  Ensuring  the  confidentiality  of 
records,  avoiding  legal  issues  related  to 
testing,  and  developing  instruments  that 
are  reliable  and  valid  are  not  easy  tasks. 

The  most  important  challenge,  however, 
is  one  of  facilitation  and  is  faced  by  the 
provider  who  in  many  instances  has  the 
primary  responsibility  for  overseeing  the 
evaluation.  Often  the  teacher  or  the  coordi- 
nator of  the  program  become  responsible 
for  managing  the  whole  planning,  imple- 
mentation, evaluation,  and  institutional- 
ization of  the  program.  These  are  not  small 
tasks  and  require  a substantial  amount  of 
skill  and  experience  in  programming  and 
working  with  different  constituent  groups. 
Even  when  teachers  and  coordinators  have 
the  know  how  and  resources  to  do  this 


work  well,  they  still  encounter  numerous 
obstacles  on  the  road  to  demonstrating 
program  effectiveness.  Limited  access  to 
information  and  key  individuals  at  the 
company  or  union  have  made  the  manage- 
ment of  the  planning  and  evaluation 
process  a tough  hurdle  to  jump. 

FROM  PRACTICE  TO  THEORY  AND 
POLICY:  THE  PLANNING  AND 
EVALUATION  TEAM  MODEL 

For  a decade,  a variety  of  federal  and 
state  funding  sources  have  supported 
workplace  education  programs  in  Massa- 
chusetts. Under  the  umbrella  of  the  Massa- 
chusetts Workplace  Education  Initiative  — 
a statewide  partnership  among  business, 
labor,  education,  and  government  (repre- 
sented by  agencies  that  fund  workplace 
education  programs)  — extensive  evalua- 
tions are  conducted  annually  to  identify 
best  practices.  A 1991  evaluation  summa- 
rized the  challenges  practitioners  faced  in 
implementing  workplace  education  pro- 
grams and  articulated  policy  recommenda- 
tions to  address  them. 

One  of  the  main  recommendations  was 
to  put  in  place  a governance  structure, 
rather  than  an  advisory  structure,  for  local 
business,  labor  and  education  collabora- 
tions. The  recommendation  acknowledged 
the  need  to  systemically  address  the 
significance  of  performing,  planning, 
implementation,  evaluation,  and  institu- 
tionalization functions  through  a team- 
based  approach.  This  approach  would 
ensure  that  clear  and  realistic  goals  for 
the  program  were  set,  agreed  upon  by  all 
partners,  and  eventually  met  by  enabling 
all  those  who  could  benefit  from  the 
program  to  participate  in  shaping  it.  The 
proposed  model  was  the  "Planning  and 
Evaluation  Team."  The  MA  Department  of 
Education  has  since  required  all  recipients 
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When  debating 
goals,  it  often 
becomes  clear  that 
team  members  may 
hold  different 
views.  To  strike  a 
balance,  realism 
and  solid  data  help 
the  team  get  clearer 
on  realistic  goals 
and  how  to  priori- 
tize them. 


of  its  workplace  education  awards  to 
implement  this  team-based  approach. 

The  initial  results  of  piloting  the  Plan- 
ning and  Evaluation  Team  (PET)  model 
were  favorable  and  currently  variations  of 
the  model  are  being  used  by  all  workplace 
education  programs  funded  by  the  Massa- 
chusetts Department  of  Education  with 
National  Workplace  Literacy  Program  and 
8%  Job  Training  Partnership  Act  funds. 
Today  there  are  27  PETs  at  sites  funded 
through  the  National  Workplace  Literacy 
Program  and  an  additional  five  at  sites 
funded  through  the  8%  Job  Training 
Partnership  Act.  PETs  consist  of  represen- 
tatives from  upper  management,  supervi- 
sors, employees  (who  are  enrolled  in  the 
program,  have  an  interest  in  enrolling,  or 
have  graduated  from  the  program),  teach- 
ers and  administrators  representing  the 
education  provider,  and  labor  representa- 
tives. Teams  meet  regularly  throughout 
the  life  of  the  program.  Initially,  teams 
may  meet  very  frequently.  As  the  program 
solidifies,  teams  meet  less  often. 

FUNCTIONS  AND  TASKS  PERFORMED 
BY  PLANNING  AND  EVALUATION 
TEAMS 

The  purposes  of  the  PET  are  multi-fold: 
to  plan  the  program,  to  oversee  implemen- 
tation, to  design  and  implement  evalua- 
tion, to  promote  improvement,  and  to 
institutionalize  and  integrate  the  program 
in  the  long-term  strategy  of  the  business 
and/or  union.  The  Massachusetts  Depart- 
ment of  Education  views  its  funding  as 
seed  funds  or  an  investment  in  building 
the  capacity  of  the  businesses  or  unions  so 
they  are  equipped  to  continue  the  program 
without  public  funding.  The  return  on  this 
investment  has  been  promising.  Almost 
65%  of  all  partnerships  that  received 
funding  under  the  National  Workplace 


Literacy  Program  through  four  funding 
cycles,  continued  when  public  funding 
ended. 

PETs  handle  specific  tasks  and  go 
through  several  stages.  First,  the  commit- 
ment from  labor  and  management  to  a 
team-based  approach  is  sought.  Subse- 
quently, a team  is  established  by  identify- 
ing key  players  in  the  company,  labor  and 
the  learning  provider.  In  this  discussion, 
the  role  of  the  team,  the  time  and  energy 
required,  and  ways  to  orient  team  mem- 
bers are  usually  clarified.  The  first  tasks 
are  usually  accomplished  in  pre-program 
discussions  between  the  education  pro- 
vider, management,  and  the  union. 

Once  the  team  is  established,  a start-up 
meeting  is  set.  The  team  begins  identifying 
goals  for  the  workplace  education  program 
by  answering  questions  such  as  "What  is 
our  ultimate  reason  for  having  the  pro- 
gram ? What  is  the  purpose  of  this  pro- 
gram? What  goals  do  we  hope  to 
accomplish  by  the  end  of  this  program? 
When  debating  goals,  it  often  becomes 
clear  that  team  members  may  hold  differ- 
ent views.  To  strike  a balance,  realism  and 
solid  data  help  the  team  get  clearer  on 
realistic  goals  and  how  to  prioritize  them. 
At  this  point,  it  is  critical  to  spell  out  how 
program  goals  relate  to  business  and  labor 
goals  and  not  to  view  them  as  fully  equal. 
Workplace  education  programs  can  help 
address  business  needs  and  goals  but  in 
few  instances  are  the  sole  incentive  needed. 

The  next  task  involves  identifying 
appropriate  indicators  that  show  the 
program  is  making  progress  towards  its 
goals.  Following  this  step,  the  PET  identi- 
fies which  components  of  the  program 
(e.g.  three  ESOL  classes)  and  what  kind 
and  level  of  business  and  labor  support 
(e.g.,  release  time)  is  needed.  This  is  also 
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the  time  when  resources  are  allocated. 

Deciding  what  to  evaluate,  how,  when, 
and  why  is  next.  The  PET  selects  or  devel- 
ops ways  to  gather  the  data  the  program 
needs;  collects  and  analyzes  the  data;  and 
utilizes  the  information  to  improve  the 
program.  In  considering  data-gathering 
instruments,  the  PET  considers  resources 
(e.g.,  time  and  dollars  required);  levels  of 
participation  required;  staff  expertise  in 
evaluating;  and  other  factors. 

Improving  the  program  based  on 
evaluation  results  may  lead  to  modifying 
goals.  This  is  the  last  task  the  PET  under- 
takes after  going  through  the  planning  and 
evaluation  cycle  once.  PETs,  however,  do 
not  necessarily  wait  until  this  stage  to 
make  revisions  to  their  goals.  Changes  in 
the  industry  (e.g.,  the  recent  restructuring 
and  refinancing  of  health  care)  may  cause 
the  team  to  modify  the  course  of  the 
program  prior  to  completing  its  planning 
and  evaluation  cycle.  Another  instance 
may  be  that  the  team  discovers  that  they 
are  collecting  important  data  that  are  not 
captured  by  its  current  goal  statements. 
These  instances  demonstrate  the  need  to 
implement  stages  and  tasks  for  PETs  on  an 
ongoing  basis  that  allows  for  flexibility  and 
ensures  that  the  process  is  non-linear. 


FROM  THEORY  TO  PRACTICE: 
IMPLEMENTATION  ISSUES 

PETs  have  been  successful  to  varying 
degrees.  More  than  five  years  of  working 
with  the  PET  model  has  taught  practitio- 
ners that  a number  of  issues  surface  year 
after  year  and  in  both  state-funded  pro- 
grams and  other  publicly-  and  privately- 
funded  programs  that  have  attempted  to 
replicate  the  PET  model.  The  issues  are: 

• How  do  we  start  up  a PET  in  a program 

that  is  not  funded  by  a public  funding 


source  that  requires  team-based  ap- 
proaches to  programming? 

• How  do  we  get  supervisors  and  other 
company  representatives  involved  in 
looking  at  program  impact  on  the 
workplace? 

• How  does  the  PET  as  a group  develop 
and  use  a set  of  tools  to  gather  data? 

• What  do  PET  members  need  to  be 
aware  of  when  developing  tools?  What 
happens  when  we  get  data  we  cannot 
use? 

• How  do  teachers  and  coordinators  on 
PETs  facilitate  identifying  goals  that  do 
not  hold  the  program  accountable  for 
the  broader  company  goals,  which  may 
require  more  initiatives  or  interventions 
than  the  program? 

Program  coordinators  from  workplace 
education  programs  at  business  sites 
across  the  Commonwealth  responded  to 
these  issues  and  identified  lessons  learned. 

2.  How  do  we  convince  an  employer  of  the 
importance  of  a governing  body  responsible 
for  developing  and  implementing  a workplace 
education  program  when  it  is  not  required  by 
a grant?  How  do  we  pull  all  the  necessary 
players  together?  Debbie  Tuler  talks  about  her 
experience  in  a Hospital  Workplace  Education 
program: 

I was  raised  on  Massachusetts'  team- 
based  approach  to  planning  and  evaluat- 
ing workplace  education  programs.  The 
data  collected  over  the  years  in  the  Massa- 
chusetts Department  of  Education  Pro- 
grams demonstrated  the  value  of  this 
team-based  approach.  So,  when  I took  over 
a private  contract  with  a hospital  (which 
had  offered  ESOL/ ABE  classes  for  five 
years  without  a team  or  advisory  board), 

I expected  that  I would  institute  the  PET 
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model  in  order  to  bring  the  program  in 
line  with  what  I believe  are  indicators  for 
effective  programs.  I also  believe  that  a 
collaborative  model  like  the  PET  approach 
is  important  because  it  keeps  players 
connected,  committed,  and  informed  about 
what  is  involved  in  a long-term  educa- 
tional process,  the  workplace  education 
program.  I did  realize,  however,  that  I 
could  not  force  it. 

First,  I had  to  convince  managers/ 
supervisors  that  it  was  worth  their  time 
and  effort  to  be  involved  in  the  program 
at  some  level.  Managers  and  supervisors 
were  unfamiliar  with  the  whole  idea  of 
having  a coordinator/instructor  speak 
with  them  about  areas  of  concern  and 
evidence  of  progress  and  to  request  docu- 
ments and  information  from  their  depart- 
ments. They  viewed  the  program  as  a 
benefit  that  the  hospital  offered  to  workers 
and  in  some  abstract  way  was  a benefit 
to  them  too,  but  they  had  never  clearly 
understood  this  benefit. 

I pursued  conversation  with  managers 
and  supervisors.  There  were  a lot  of 
changes  taking  place  in  the  hospital.  This 
meant  that  employees  would  have  an 
opportunity  to  upgrade  their  position  if 
their  language  skills  improved  and  if 
supervisors  and  managers  noticed  that 
their  language  skills  had  improved.  It  was 
important  to  get  supervisors  and  managers 
to  see  the  benefits  of  the  program  to  them 
and  to  the  workplace  in  general.  By  asking 
questions  and  engaging  supervisors  and 
managers  in  conversations  we  encourage 
them  to  recognize  the  benefits. 

By  encouraging  managers  and  supervi- 
sors from  Environmental  Services,  Trans- 
portation and  Nutritional  Services  at  the 
hospital  to  offer  input  into  the  program,  I 
did  in  fact  open  the  door  for  expressing 
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concerns.  Until  now,  they  had  mainly 
voiced  frustrations  that  more  workers  from 
the  day  shift  do  not  take  advantage  of  the 
benefit  of  classes  (classes  met  from  3-5  pm 
and  their  shifts  end  at  3:30).  At  the  same 
time,  supervisors  can  only  let  so  many 
people  attend  class  at  the  same  time  during 
work  hours  because  of  staffing  constraints. 
Students  pointed  out  to  me  that  other  day 
shift  workers  cannot  attend  class  because 
they  had  other  jobs  or  child  care  concerns. 
Issues  of  concern  then  related  to  schedul- 
ing, recruitment  and  policies.  I pointed  out 
to  each  party,  supervisors  and  students, 
that  it  would  be  easier  to  resolve  these 
issue  if  everyone  sat  at  the  table  together 
and  discussed  these  issues  rather  than 
having  me  be  an  intermediary. 

Last  spring  (1996),  after  six  months  of 
working  at  the  hospital  and  getting  to 
know  the  various  players,  I was  able  to 
get  commitments  on  the  part  of  managers 
from  Environmental  Services,  Nutritional 
Services,  and  Transportation  to  participate 
in  an  "Education  Committee"  (a  PET  by 
another  name)  which  would  include 
workers  in  the  program.  Their  task  would 
be  to  address  some  of  the  issues  mentioned 
above.  We  would  start  there  and  move  to 
other  areas  of  the  program  where  this  col- 
lective input  is  needed,  including  curricu- 
lum design  and  goal  setting. 

We  had  our  first  meeting  in  October  to 
air  and  clarify  concerns  and  to  identify 
next  steps.  After  discussing  the  reasons 
people  cannot  come  to  class  and  the  issues 
surrounding  releasing  workers  to  attend 
class,  we  came  to  some  decisions.  These 
changes  amounted  to:  change  the  time  of 
day  of  classes  in  order  to  encourage  more 
participation  from  day  shift  workers,  have 
the  instructor  attend  departmental  staff 
meetings  to  promote  the  program  with 
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assistance  from  current  and  past  students, 
and  set  up  a buddy  system  where  workers 
who  attend  the  program  could  bring 
another  worker/ friend  to  class  to  try  it  out. 

What  impressed  me  about  the  meeting 
was  that  people  were  talking  to  each  other, 
not  just  me.  I also  got  a commitment  from 
the  committee  to  meet  monthly  to  resolve 
issues  that  arise.  I felt  that  communication 
was  opening . With  this,  though,  came  the 
realization  that  the  managers  believed  that 
the  programs  solely  benefited  the  workers 
who  participated  in  it  and  not  themselves. 
The  meeting  did  little  to  change  this 
attitude  "They  should  take  advantage  of 
this. . .they  should  make  time  to  improve 
their  language  skills".  For  the  program  to 
continue  and  develop,  this  perception 
needs  to  change.  This  can  be  done  in  the 
committee  or  with  individual  managers.  It 
is  important  to  keep  asking  questions  and 
engaging  in  dialogue.  They  need  to  buy 
into  the  program  so  that  they  argue  for 
its  continuation  amidst  budget  cuts  and 
layoffs. 

My  experience  at  the  hospital  during  the 
past  nine  months  has  made  it  clear  to  me 
that  PETs  (or  their  equivalents)  in  different 
workplaces  may  play  different  roles  and 
define  their  purposes  in  different  ways. 
Defining  these  roles  and  purposes  is  a real 
challenge. 

2.  How  do  we  get  supervisors  and  other 
company  representatives  involved  in  looking  at 
program  impact  in  the  workplace?  And  why  is 
this  important?  Jane  Shea  from  Quinsigamond 
Community  College's  workplace  education 
programs  talks  about  taking  involvement  of 
supervisors  and  other  company  personnel  to 
another  level: 

It  really  all  starts  with  the  initial  goal 
setting  process  that  we  do  with  our  plan- 


ning and  evaluation  teams.  This  is  our 
springboard.  We  ask  the  PET  to  help 
identify  people  within  their  workplace  to 
solicit  feedback  about  the  progress  of  the 
program.  We  have  them  identify  people 
who  they  think  are  not  that  knowledgeable 
about  the  program  and  its  benefits,  as  well 
as  those  who  are  directly  involved  through 
the  PET,  supervisors  of  workers  etc. 

Most  people  tend  to  respond  to  ques- 
tions about  program  impact  and  progress 
by  talking  about  individual  students.  We 
make  it  clear  that  we  design  our  questions 
to  help  people  look  more  broadly  at  the 
program  s impact  and  less  on  individual 
students.  This  process  takes  time;  it  also 
involves  asking  the  questions  over  and 
over  and  engaging  in  dialogue.  For  ex- 
ample, we  ask,  'Have  you  seen  any 
changes  in  the  workplace/ workers?  What 
do  you  hope  to  see  in  this  workplace  after 
the  program  has  ended? 

Of  course,  getting  people  to  answer 
questions  can  be  a difficult  task.  People 
are  generally  very  busy  and  hard  to  track 
down  so  we  need  to  be  flexible  in  soliciting 
this  information.  As  the  coordinator,  I will 
go  to  staff  meetings  and  use  the  questions 
to  gather  information  collectively.  I will 
ask  people  when  I happen  to  see  them  in 
the  hall.  I will  send  questions  out  to 
individuals  in  a survey  form  and  then 
arrange  an  appointment  to  meet  them 
and  discuss  the  questions.  I have  also 
made  appointments  without  giving  out 
the  questions  in  advance. 

It  is  worth  finding  a variety  of  methods 
to  gather  this  information  in  order  to  note 
the  many  different  perspectives  within  the 
company.  The  ideas  and  comments  we  get 
from  supervisors  and  workers  who  have 
direct  or  indirect  impact  on  the  program  is 
very  valuable.  They  deepen  our  under- 
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standing  of  the  workplace.  These  com- 
ments inform  us  about  the  employees' 
perception  of  the  program. 

Of  course  there  is  always  the  question 
of  confidentiality  for  students  as  well  as 
supervisors.  We  are  very  clear  up  front 
before  we  start  working  with  programs, 
that  we  have  a clear  policy  protecting  the 
confidentiality  of  individual  students  in 
the  program  and  those  that  have  offered 
to  answer  the  evaluation  questions.  We 
realize  that  it  is  very  hard  when  you  ask 
someone  how  they  think  the  program  is 
going  for  them  not  to  think  about  an  indi- 
vidual student  in  particular.  The  questions, 
as  I mentioned  before  are  designed  to  look 
at  the  general  and  not  specific. 

But  here  is  the  dilemma:  we  really  like 
hearing  and  sharing  about  the  successes  of 
students  or  the  positive  feedback  of  super- 
visors and  managers  and  really  have  to 
try  to  hold  to  the  same  principles  and  give 
that  worker  the  opportunity  to  decide 
whether  to  disclose  the  success,  such  as 
getting  a GED  or  not. 

Although  people  within  the  program 
(learners,  teachers,  PET  members)  aid  in 
the  data  collection  process,  PET  has  the 
responsibility  to  report  that  information  to 
others  in  a way  that  recognizes  and  vali- 
dates the  supervisor's  voice,  maintains  a 
level  of  confidentiality  for  the  workers 
participating  in  the  program,  and  gives 
enough  information  to  help  inform  the 
planning  process. 

3.  How  does  the  PET  as  a group  develop  and 
use  a set  of  tools  to  gather  data?  John 
Antonellis,  coordinator  of  Workplace  Educa- 
tion programs  at  Jewish  Vocational  Programs 
in  Boston,  talks  about  where  tools  may  come 
from: 

The  starting  point  for  developing 

W3 


evaluation  tools  is  the  program's  goal 
statement.  Usually,  the  PET  will  examine 
the  goals  for  the  program  and  discuss  who 
is  in  the  best  position  to  evaluate  each  goal. 
At  this  point  we'll  have  a brainstorm  about 
the  kinds  of  tools  that  could  be  used  to 
gather  the  information  we  need.  Next,  I'll 
draft  a set  of  tools  and  bring  these  back  to 
the  PET  at  the  next  meeting.  Having  some- 
thing concrete  to  work  with  is  important. 
I've  found  that  this  makes  it  very  clear  if 
we're  all  talking  on  the  same  wavelength. 
The  education  provider  is  the  perceived 
expert  regarding  program  evaluation  and 
does  in  fact  have  access  to  a wealth  of 
resources  based  on  past  experience  which 
can  be  used  to  develop  evaluation  tools 
that  are  tailored  to  each  program.  Further, 
as  most  PET  representatives  are  under 
pretty  serious  time  constraints,  coming  to 
the  PET  with  a draft  in  hand  is  a more 
efficient  use  of  time.  The  PET  then  looks 
over  the  drafts  and  any  needed  changes 
are  made. 

Embedded  in  every  evaluation  tool  are 
assumptions  about  your  program's  goals. 
Ideally,  the  process  described  above  occurs 
early  on  in  the  establishment  of  a work- 
place program.  While  programs  usually 
develop  a goals  statement  early  on  in  the 
process,  they  have  not  always  linked 
their  evaluation  tools  explicitly  to  these 
goals.  In  some  cases,  the  program  may 
be  using  tools  developed  for  a previous 
program  or  may  have  been  developed 
independently  of  the  goals  statement.  In 
cases  like  this,  what  I do  is  examine  both 
the  goals  statement  and  the  existing 
evaluation  tools.  Together  with  the  PET  we 
attempt  to  make  the  connection  between 
the  goals  statement  and  evaluation  tools 
explicit..  In  some  instances  we  have  found 
that  the  goals  embedded  in  an  evaluation 


VENTURES  IN  ASSESSMENT 


Volume  9:  December  1996 


tool  more  accurately  reflect  the  true 
program  goals,  and  the  goal  statement  is 
adapted  accordingly.  In  other  instances 
new  tools  are  developed  to  account  for 
any  gaps  found  between  the  desired 
program  goals  and  existing  evaluation 
tools.  My  experience  has  been  that  pro- 
grams are  generally  very  good  at  design- 
ing tools  which  measure  the  impact  of  the 
program  on  the  participant,  but  measuring 
the  impact  on  the  workplace  and  on  the 
educational  partnership  is  a greater  chal- 
lenge. 

We  are  really  trying  to  develop  tools 
which  provide  both  qualitative  and 
quantitative  data  that  is  used  to  improve 
existing  programs  and  aid  them  in  their 
efforts  toward  institutionalization.  One  of 
our  most  successful  tools  is  the  survey 
because  it  can  be  designed  to  provide  both 
quantitative  and  qualitative  information, 
it  is  relatively  easy  to  administer,  and 
requires  only  a minimum  of  time  to 
complete.  The  surveys  I've  developed  are 
explicitly  tied  to  the  goal  statement,  are 
short  (one  side  of  one  page),  and  include 
a combination  of  open  and  closed  ended 
questions.  Open  ended  questions  provide 
qualitative  information  while  closed  ended 
question  provide  quantitative  results. 
Another  advantage  of  using  surveys  as  an 
evaluation  tool  is  that  the  same  survey  can 
usually  be  adapted  for  use  by  both  super- 
visors and  program  participants.  So 
everyone  is  evaluating  the  program  for  the 
same  goals. 

4.  What  happens  when  we  get  data  we  cannot 

use? 

I think  I have  heard  this  joke  before,  and 
the  punch  line  is  " manipulate  the  data" 
Wait!  that  was  intended  as  humor  and  not 
a suggestion. 

0 
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Most  data  can  be  used  in  " legitimate" 
descriptive  ways  even  if  that  isn't  really 
what  had  been  planned.  Any  data  collec- 
tion plan  should  include  a section  on  how 
you  intend  to  use  the  data.  Potential  " use" 
problems  often  can  be  detected  at  the  plan- 
ning stage.  However,  if  you  forgot  this 
step  in  the  plan,  relax  and  think  creatively 
( and  legitimately). 

For  instance,  you  may  have  hoped  to 
discover  how  many  students  progressed 
during  their  time  in  your  course,  but  you 
only  have  aggregate  data  indicating  class 
pre-test  scores  of  10, 10, 20,  20, 30,  and  30 
and  post-test  scores  of  10, 20, 20, 30, 30, 
and  30.  In  this  example,  one  10  could  have 
moved  to  30  (one  person  progressed);  one 
10  could  have  moved  to  20  and  one  20 
moved  to  30  (two  people  progressed);  or 
two  10's  could  have  moved  to  20,  one  20 
moved  to  30,  and  one  20  regressed  to  10 
(three  people  progressed);  etc.  You  can't 
"use  this  data"  to  indicate  how  many 
students  progressed,  but  you  can  use  this 
data  to  state  that  the  "class  average  post- 
test" (23.3)  is  3.3  points  (or  17%)  higher 
than  the  " class  average  pre-test"  (20). 

This  is  a classic  "unit  of  analysis"  problem: 
you  wanted  student  (unit)  information, 
but  because  you  didn't  track  individual 
student  progress,  you  got  class  (unit) 
information.  Actually,  I think  a 17% 
overall  increase  sounds  better  than  saying 
one  or  two  students  progressed. 

Most  data  that  is  collected  for  loftier 
purposes  (e.g.  unearthing  correlations)  can 
be  very  useful  as  descriptive  if  your  data 
really  doesn't  meet  statistical  assumptions. 
Most  readers  understand  and  enjoy 
pictures  (descriptions)  as  much  as  statistics 
anyway. 

A word  of  caution  when  thinking 
creatively  about  your  data:  in  some  cases 
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your  lemons  cannot  be  turned  into  lemon- 
ade. If  you  really  have  garbage  data  (check 
list  below  for  suggestions  on  getting 
garbage-proof  data),  don't  try  to  recycle 
it.... JUST  TOSS  IT  OUT. 

Hints  for  effective  evaluations.  These 
are  suggestions  that  I find  helpful  in 
guiding  my  evaluation  work. 

1.  QUERY.  Start  with  a specific,  simply- 
phrased  research  question,  such  as:  Do 
students  in  the  Math  classes  make  fewer 
computational  errors  on  sample  work 
orders  after  the  course  than  before?  The 
question  identifies  WHO  (students  in  the 
Math  Class),  WHAT  (computation),  HOW 
(sample  work  orders),  and  WHEN  (before- 
and-after-course  participation)  measure- 
ment will  occur.  ' 

2.  CLARIFY.  Try  to  identify  factors  that 
might  impact  the  answer  to  your  question, 
other  than  the  factor  you  would  expect. 

For  the  question  above,  the  style  of  instruc- 
tion may  make  a difference  in  outcomes  if 
there  are  more  than  one  class/instructor. 
The  number  of  times  (or  time  of  day) 
students  attend  the  class  could  impact 
outcomes.  Each  of  these  factors  may 
require  additional  clarifications  to  your 
question,  such  as:  'Do  students  attend  90% 
of  the  math  classes. . .'  etc.  These  clarifica- 
tions will  guide  your  analysis  and  inter- 
pretation of  the  findings. 

3.  SIMPLIFY.  Simple  data  collections, 
such  as  a test  or  survey,  may  not  be  as 
soul-satisfying  as  extensive  interviews  or 
simulations,  but  they  are  usually  easier  to 
do  and  yield  easily-interpreted  data.  If 
you  choose  a more  complex  route  to  data 
collection,  be  sure  you  plan  for  the  extra 
time  that  will  probably  be  needed  to 


analyze  the  results.  For  example,  open- 
ended  interviews  often  yield  very  rich 
information,  but  analyzing  this  data  can 
require  several  readings,  coding  and 
reliability  checks. 

4.  QUANTIFY.  Anecdotal  evidence 
enhances  evaluations,  but  most  businesses 
and  funders  need  quantified  results.  Even 
data  that  is  collected  qualitatively,  through 
interviews  or  open  ended  questions, 
should  be  coded  into  categories  and 
quantified. 

5.  UTILIZE.  Even  if  you  are  doing  evalua- 
tion to  satisfy  a requirement  of  your 
employer  or  funder,  make  sure  it  is  useful 
to  your  program  and  that  you  USE  the 
information  you  get.  Evaluation  takes  time 
and  energy,  and  GOOD  evaluation  is  an 
excellent  tool  for  program  improvement. 

5.  How  do  teachers  and  coordinators  on  a PET 
facilitate  identifying  goals  that  do  not  hold  the 
program  accountable  for  the  broader  company 
goals,  which  may  require  more  initiatives  or 
interventions  than  just  the  program? 

It  is  crucial  during  the  goal  setting 
process  to  phrase  questions  in  such  a way 
as  to  help  employers  see  the  distinction 
between  company  goals  and  program 
goals.  When  an  employer  says  that  his  goal 
is  to  improve  productivity,  ask  how  the 
program  can  help  you  achieve  this  goal. 
What  is  the  link  between  the  basic  skills 
instruction  which  we  offer  and  the 
company's  broader  goals  of  improving 
productivity?  Are  those  who  will  partici- 
pate in  the  program  the  ones  who  are 
responsible  for  this  desired  outcome?  Who 
else  is  responsible  but  will  not  be  partici- 
pating in  the  program,  and  how  will  we 
get  data  from  them? 
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Too  often  I have  seen  increased  reading 
and  writing  skills  as  a desired  outcome  for 
participants  when  they  are  not  in  a posi- 
tion that  requires  any  visible  reading  and 
writing  or  are  often  not  offered  opportuni- 
ties to  show  gains  in  these  areas.  The  result 
is  that  we  as  a team  do  not  have  the  data 
needed  to  determine  if  the  company  goal 


is  being  reached.  The  answers  to  these 
questions  above  will  force  the  team  to 
define  the  connection  between  the  pro- 
gram and  the  company,  and  articulate  the 
parameters  of  the  program.  This  question- 
ing should  be  the  starting  point  for  setting 
goals  and  designing  instruments. 

Off 
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Learning  from  Experience: 
The  Native  Language  Literacy 
Screening  Device 


by 

Deborah 

Mercier-Cuenca 


Salem  Harbor  Community 
Development  Corp. 
Salem,  MA 


The  Native  Language  Literacy 

Screening  Device  (NLLSD)  is  great. 
I first  used  it  several  months  ago 
with  students  who  were  already  enrolled 
in  my  Native  Language  Literacy  class.  At 
that  time,  the  class  worked  on  their  read- 
ing and  writing  in  Spanish,  with  math 
and  science  of  course  also  being  taught 
in  Spanish.  On  the  NLLSD  form  it  asks 
students  very  broadly  to  write  about  their 
families,  why  they  want  to  study  English 
and  something  about  their  lives.  This  gives 
the  students  the  opportunity  to  write  about 
anything. 

Most  of  the  more  advanced  students 
expressed  their  desire  and  immediate  need 
to  learn  English  for  a variety  of  reasons: 
employment  so  that  they  can  better  pro- 
vide for  their  family;  to  help  their  children 
with  their  school  work;  and  to  be  more 
independent  in  their  daily  tasks  by  relying 
less  on  translators. 

The  students  were  getting  impatient 
with  waiting  to  enter  ESOL  classes.  They 
were  tired  and  bored  with  reading  and 
writing  exclusively  in  Spanish.  They 
wanted  to  learn,  for  example,  practical 
phrases  they  could  use  on  a daily  basis. 
This  made  perfect  sense  to  me  but  up  until 
that  point  I felt  pretty  committed  by  our 
former  program  design  to  give  instruction 
exclusively  in  Spanish.  But  aifter  a good 
90%  of  the  students  expressed  such  interest 
in  wanting  to  learn  English,  via  the  Native 
Language  Literacy  Screening  Device,  I 
decided  to  ignore  my  fears  and  heed  my 
students'  wishes.  Thus,  to  answer  the 


question  "How  have  I used  the  NLLSD?" 

I have  used  it  to  inform  instruction  and 
through  this  decision  I have  empowered 
my  students. 

I have  used  it  for  on-going  assessment 
and  just  a week  ago  I used  it  again  for  on- 
going and  initial  assessment  (the  latter 
obviously  for  my  newer  students).  I like 
the  device  because  I can  glean  information 
about  my  students  that  I would  not  neces- 
sarily have  had  or  learned  otherwise.  I 
would  not  necessarily  have  concentrated 
my  focus  on  written  questions  about  their 
families  or  why  they  wanted  to  learn 
English  or  what  they  liked  to  do  in  their 
spare  time.  The  last  essay  they  are  asked 
to  write  has  and  will  continue  to  inform 
instruction.  It  has  helped  me  to  better 
understand  their  specific  family  issues, 
and  to  be  more  sensitive  to  the  students' 
desire  to  learn  English. 

One  student  who  recently  completed 
the  NLLSD  spoke  only  of  her  love  for 
Bingo  and  the  friends  with  whom  she 
plays  (she  had  an  endless  list  of  names). 
That  was  a real  eye  opener  for  me.  She  is 
always  talking  about  how  important  it  is 
to  speak  and  understand  English  but  since 
she  cannot  join  the  workforce,  her  needs 
are  a bit  different  — perhaps.  We  will  now 
start  playing  alphabet  (and  the  traditional) 
Bingo  again  on  a more  frequent  basis  to 
meet  her  needs. 

The  students  unanimously  agree  that 
this  new  idea  of  giving  some  instruction  in 
English  and  some  in  Spanish  is  working. 

- They  are  pleased  that  I was  in  a position  to 
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truly  listen  to  them.  Their  attendance  has 
improved  since  I started  giving  them 
instruction  in  English  and  they  are  becom- 
ing more  responsible  about  calling  and/or 
stopping  by  when  they  cannot  make  it  to 
class. 

The  essay  component  of  the  device  also 
helped  me  see  their  grammar  deficiencies 
and  strengths  in  their  native  Spanish.  My 
being  fluent  in  their  language  has  and  will 
continue  to  enable  me  to  inform  instruc- 
tion. I also  enjoyed  reading  the  four  pages 
that  preceded  the  tool.  The  information 


was  useful;  it  talks  about  how  to  use  the 
instrument  and  what  information  you  can 
conclude  from  the  results  of  the  NLLSD. . 

The  Native  Language  Literacy  Screening 
Device  is  available  in  Arabic,  Chinese, 
French,  Haitian  Creole,  Italian,  Polish, 
Russian,  Spanish,  Vietnamese,  Portuguese 
and  Tigrinya.  It  is  available  through  the 
New  York  State  Education  Department. 
Attn:  Dr.  Patricia  Mooney  Gonzalez,  Office 
of  Workforce  Preparation  and  Continuing 
Education,  Room  307  EB,  West  Wing, 
Albany,  New  York  12234. 
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Looking  down  the  road,  and  hoping  to  widen  it  for  all 

Why  ABE  Math  Assessment  Practices 
Mnst  Change 


Editor's  Note:  This  article  is  reprinted  from  The  Problem  Solver,  a publication  jointly 
funded  by  SABES  and  the  Department  of  Education.  The  edition  from  which  this 
article  was  taken  was  designed  to  introduce  and  reinforce  changes  in  mathematical 
assessment  practices  as  encouraged  by  the  Massachusetts  Curriculum  Frameworks 
document  (primarily  the  ABE  portions).  The  Problem  Solver  pages  include  materials 
developed  by  the  three  Frameworks  Math  Assessment  Study  Groups  from,  Boston, 
The  Northeast  and  West  Regions.  Much  of  the  edition  focused  on  potential  assess- 
ment tasks  and  actual  Frameworks-based  assessments  performed  as  part  of  the  study 
group  exploration. 


by 

Tricia  Donovan 


Franklin  Hampshire 
Employment  and 
Training  Consortium 
Greenfield,  MA 


Assessment  is  a key  component  of 
curriculum.  It  initiates  the  aca- 
demic teacher-student  relation- 
ship and  guides  instructional  decisions  in 
the  best  of  circumstances.  Currently,  the 
Massachusetts  Curriculum  Frameworks, 
encompassing  the  Mass.  ABE  Math  Stan- 
dards and  the  NCTM  Math  Standards, 
calls  for  teachers  to  heighten  their  aware- 
ness of  curriculum  decisions  which  may 
have  become  or  can  become  automatic, 
almost  unconscious  and  too-often  homog- 
enized to  serve  institutional  needs,  not 
student  needs.  Teachers  are  asked  in  the 
Frameworks  to  reform  their  mathematical 
teaching  practices,  including  their  assess- 
ment methodologies. 

The  Frameworks  and  Standards  beg 
teachers  to  open  up  their  questions, 
integrate  academic  skills  and  pique  criti- 
cal/creative thinking  in  their  courses,  their 
teaching.  And  all  these  changes  are  urged 
in  the  belief  that  students  can  be  better 


served  by  teachers  familiar  with  the 
nuances,  ambiguities  and  potentials  of 
mathematics.  However,  the  Frameworks 
recognize  that  it  is  impossible  to  illuminate 
a course  of  study  without  accurate,  in- 
formed and  comprehensive  assessment  of 
student  knowledge  and  learning  which  by 
rights  should  drive  the  curriculum. 

A teacher  must  know  what  mathematics 
each  student  understands  in  order  to  serve 
well  that  student.  To  figure  out  what  is 
known  and  what  is  not,  how  a person 
learns  and  what  blocks  his  or  her  learning, 
requires  assessment  of  a sort  far  different 
from  that  which  has  traditionally  been 
practiced;  it  requires  an  'assessment 
mindset'  that,  yes,  utilizes  new  tools,  but 
more  importantly  is  exercised  diligently/ 
insightfully,  regardless  of  the  problem,  test 
or  task  used.  The  evolution  in  assessment 
asks  teachers  to  think  deeply  about  math- 
ematics and  about  students  all  the  time. 

"Every  mathematical  activity  can 
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become  an  opportunity  for  assessment," 
note  teachers  engaged  in  exploration  of 
Frameworks-based  assessment. 

Both  the  Frameworks  and  the  NCTM 
Standards  champion  an  evolution  in 
assessment  practices,  an  evolution  in 
which  careful,  criteria-based  analysis  and 
collections  (portfolios)  of  students'  work, 
not  just  a compilation  of  their  percentages 
or  positions  on  the  bell  curve,  becomes  the 
standard  procedure  used  to  determine 
what  a student  knows,  at  what  level  (s)he 
comprehends  what  is  known  and  what 
scholastic  attitudes  or  intellectual  behav- 
iors are  exhibited.  This  evolution  assumes 
an  awareness  of  lesson  objectives  and  an 
investigation  of  students'  meeting,  exceed- 
ing or  confusion  with  these  objectives  in 
order  to  inform  subsequent  instruction. 

Frameworks-based  assessment  requires 
teachers  to  know  what  it  is  they  expect 
students  to  learn  from  or  demonstrate  via 
a particular  lesson.  Random  selection  of 
activities,  whether  to  keep  students  busy 
or  to  entertain  them,  is  viewed  as  counter- 
productive. A clear  vision  of  students' 
knowledge  base  and  direction  for  future 
development  is  required.  That  is,  the 
teacher  must  know  the  student  and  the 
subject  in  order  to  develop  or  locate 
appropriate  materials  for  instruction 
and  assessment. 

When  assessing  students,  teachers 
are  advised  to  ask: 

What  skills,  knowledge,  and  academic 
behaviors  do  I want  my  students  to 

express,  and  what  tasks  will  provide 
evidence  of  these? 

The  emphasis  here  is  on  establishing 
criteria  to  prove  mastery.  Does  the  student 
understand  concepts,  techniques,  algo- 


rithms, etc.,  and  can  (s)he  apply  this 
knowledge  flexibly  and  appropriately, 
and,  finally,  what  criteria  will  establish  for 
teachers  valid  proof  that  a student  knows 
the  concepts,  techniques,  algorithms,  etc. 
or  possesses  academic  habits  of  thinking? 

Documentation  of  investigation  results 
is  necessary,  just  as  recording  numbers 
and  case  notes  has  been.  Only  now  what  is 
recorded  is  information  relative  to  specific 
curriculum  objectives,  especially  as  articu- 
lated in  the  Frameworks  and  the  Stan- 
dards. Is  the  student  a flexible  problem 
solver?  Can  the  student  coherently  com- 
municate the  mathematics  embedded  in  a 
problem?  Does  the  student  apply  previous 
knowledge  to  a new  situation?  The  an- 
swers to  these  questions,  as  demonstrated 
on  particular  tasks  or  in  exploration 
projects  or  in  portfolios,  are  the  informa- 
tional foundations  of  Frameworks-based 
academic  assessment.  (It  must  be  noted 
that  academic  assessment  always  operates 
within  the  larger  gestalt  of  assessment 
which  for  devoted  teachers  involves 
assessment  of  students'  physical,  psycho- 
logical and  spiritual  statuses  as  well  as 
mental  or  academic  status.)  These  informa- 
tional foundations  are  built  to  help  teach- 
ers better  serve  the  academic  needs  of  their 
students. 

And  it  is  assumed  that  every  ABE 
(Adult  Basic  Education)  teacher  wants  to 
serve  her  or  his  students  in  the  best  man- 
ner possible.  Every  adult  educator  wants 
his  or  her  students  to  be  successful,  to  get 
what  "they  came  for"  and  to  feel  better 
about  themselves  as  they  become  more 
confident  in  their  abilities.  These  are  the 
ABE  teacher's  rewards.  Moreover,  all  ABE 
instructors  know  the  bottom  line  for  many 
funding  sources  and  the  basic  goal  of  most 
students  is  attainment  of  the  GED  certifi- 
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cate.  The  GED  certificate  is  accepted  as 
recognition  of  general  academic  achieve- 
ment. But  for  too  many  students  achieve- 
ment of  the  GED  with  a minimum  score 
in  mathematics  leads  to  educational  and 
economic  stagnation.  It  is  not  sufficient 
preparation  for  what  comes  later.  Mini- 
mum competency  is  not  enough;  it  is  not 
serving  students  in  the  best  manner 
possible. 

By  aligning  itself  with  the  K-12  teachers 
and  administrators  who  are  developing 
Curriculum  Frameworks  and  Alternative 
Assessment  Practices,  the  Massachusetts 


ASSESSMENT  DEFINITIONS 

Webster's:  appraisal  (expert  judgment  of  value  or  merit);  estimation; 
to  determine  tfie  amount  or  rate  of  something;  evaluate  (value  or 
judge) 

SABES  Alternative  Assessment  Project:  "The  whole  (alternative 
assessment)  experience  has  been  one  of  getting  to  know  each  other." 
Paul  Trunnell 

Teaching  Exceptional  Children:  Special  Needs:  The  "process  of 
collecting  information  about  individuals  or  groups  for  the  purposes  of 
making  decisions."  Ordinarily  refers  to  testing,  observation  and 
interviewing.  Results  should  help  decide  whether  problems  exist  and  if 
so,  what  to  do. 

NCTM:  'The  process  of  gathering  evidence  about  a student's  knowl- 
edge or,  ability  to  use  and  disposition  towards  mathematics  and  of 
making  inferences  from  the  evidence  for  a variety  of  purposes."  This 
process  of  describing  what  students  know  and  do  should  be: 

• aligned  with  and  integral  to  instruction 

• from  multiple  sources 

• use  methods  appropriate  for  purposes 

• check  all  aspects  of  math  knowledge  (investigation,  formulation, 
representation,  reasoning,  applying  a variety  of  strategies  and 
shifting  from  mechanical  to  cognitive  skill  comprehension. 


Adult  Education  community  seeks  to 
dream  a bigger  dream  for  ABE  students, 
one  which  unlocks  the  prison  of  poverty 
and  social  disenfranchisement.  It  seeks  to 
clear  paths  to  economic,  social  and  per- 
sonal growth  which  formerly  opened  only 
for  those  prepared  by  a college-prep  or 
advanced  technological  prep  high  school 
program.  It  says  getting  by  by  the  skin  of 
your  teeth  is  not  enough.  It  says,  "We  can 
help  you  learn  to  solve  problems,  imagine 
problems  and  discuss  them  with  others." 

The  purpose  of  ABE  is  not  only  to  teach 
students  a few  tricks  to  get  past  the  'aca- 
demic guards'  in  the  GED  Testing  Office; 
it  is  to  awaken  learners  who  are  alienated 
from  the  unavoidable  world  of  academic 
ideals,  and  arouse  those  beginning  to  see 
the  value  of  and  power  in  the  ability  to 
read  all  genres,  write  coherently,  calculate 
and  problem  solve  with  mathematics. 

To  awaken  and  arouse  learners  requires 
ongoing,  informed,  documented  assess- 
ment of  learning.  Armed  with  such  assess- 
ment information,  every  teacher  becomes 
capable  of  leading  students  down  their 
various  paths  to  success. 

The  paths  to  achievement,  economic 
equality  and  personal  development  are 
being  widened  by  efforts  like  the  Frame- 
works so  people  of  all  backgrounds  can 
walk  down  them.  Across  the  State,  teach- 
ers and  administrators  seek  ways  to  better 
prepare  all  students  for  the  challenges 
known  to  lie  ahead.  These  educational 
leaders  realize  that  successful  citizens  are 
able  to  reason,  solve  problems,  communi- 
cate and  make  connections  throughout 
their  lives.  They  know  that  when  a 
student's  creative  intelligence  and  curios- 
ity are  allowed  to  flourish,  that  student 
plants  her/his  feet  firmly  on  the  paths  to 
fulfillment  and  success.  The  determination 
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of  the  Massachusetts  Curriculum  Frame- 
works and  the  development  of  Frame- 
works-based  assessment  practices  are 
efforts  to  help  us  all  become  better  skilled, 
sharper  thinking  citizens  who  are  able  to 
question  and  develop  solutions  to  prob- 
lems. 

Teacher  and  student  reflection  upon 
classroom  work  is  integral  to  the  process  of 
improvement.  The  quality  of  thinking,  the 
quality  of  interaction,  the  quality  of  expres- 
sion of  each  student  must  be  considered 
and  then  evaluated  in  order  for  learning  to 
take  place.  In  this  process  of  observation, 

• i_! i! n _ * j - 

trAOJULLiiaLiuii,  active  leiiecuuu,  tunsiuei- 

ation  and  evaluation  which  The  Frame- 
works terms  assessment,  the  intention  is 
to  better  serve  the  individuals  involved  in 
educational  development.  Percentages, 
bell  curve  positions  and  vague  intuitions 
are  not  enough  to  prepare  adult  students 
of  basic  education  for  the  demands  of  the 


21st  century.  Teachers  need  to  know  what 
students  truly  understand  and  how  they 
came  to  understand  it  if  they,  the  teachers, 
desire  to  nourish  the  students'  intellectual 
capacities  and  free  them  from  the  con- 
straints of  a limited  education. 

To  improve  students'  abilities  to  maneu- 
ver in  a fast-changing  world,  to  broaden 
their  horizons  and  stimulate  the  desire  to 
understand  more  about  themselves  and 
that  world  which  surrounds  and  influ- 
ences them,  teachers  have  to  listen  and 
look  deeply,  discerningly,  at  students' 
mathematical  utilizations  and  inventions. 
Tills  is  what  FramewOrks-baSed  assess- 
ment advocates.  This  Frameworks-based 
assessment  process  is  neither  easy  nor 
simple,  but  it  promises  to  clarify  teachers' 
perceptions  of  student  and  to  lead  to  more 
accurate  and  useful  evaluations  of  stu- 
dents' accomplishments  and  needs. 


To  awaken  and 
arouse  learners 
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informed, 
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ASSESSMENT  DISCUSSION  — WHO,  WHAT,  HOW  AND  WHY 


Before  beginning  its  experiments  in  Frameworks-based 
assessment , the  West  Group  considered  basic  questions 
in  assessment  such  as  "Why  do  assessment?",  "When?", 
How  does  one  assess?",  "Who  performs  assessment ?", 
and  "What  is  being  assessed ?"  Below  is  a brief  synopsis 
of  'answers'  brainstormed  by  the  group. 

Why  and  When 

• To  place  people;  enroll  or  refer 

• Determine  learning  disabilities  or  learning  styles 

• Determine  where  help  is  needed 

• Requirements  (funding  and  program  efficacy) 

• Inform  instruction 

• To  see  what  has  been  learned 

• To  build  confidence 

• To  have  a reality  check 

• To  Determine  aptitudes/  strengths 

• PRE,  ONGOING,  POST 
Who 

• Teachers 

• Administration  (agency,  state,  fed.) 

• Society 

• Students  (self  or  peer) 


How 

• Standardized  instruments  (GED  pre-tests,  ABLE,  TABE, 
etc.) 

• Dialogue  — informal  between  students  or  between 
teacher  and  student 

• Observation 

• Thought  protocol  (formal) 

• Projects  (long  term  or  short  term) 

• Journal  reflections 

• Learning  logs 

• 'Tasks'  (problems) 

• Worksheets 

What 

• Thinking  processes:  inductive,  deductive,  logical, 
intuitive,  linear,  non-linear,  confused,  scattered, 
perseverant 

• Learning  styles 

• Knowledge:  algorithm  mastery;  concepts  such  as 
denominator,  ratio/ proportion,  etc.;  assumed  general 
knowledge  such  as  weeks  in  a year;  math  facts  (both 
recalling  and  applying) 

• Comprehension  — basically  language 

• Problem  Solving  — flexibility,  group  process,  critical 
thinking,  application,  extension 
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What  Counts 

Graphing  the  Average  Rent 
of  Athol,  Orange,  and  Greenfield 

The  following  problem  was  presented  to  a GED  class  in  Athol  by  Peg  Reidester  of 
F/HETC  who  subsequently  wrote  an  analysis  of  the  task  context  and  outcome  as 
well  as  an  assessment  of  one  student,  Esther,  using  the  Student  Assessment  Rubric 
developed  by  a Frameworks  group  in  western,  MA  of  which  she  was  a part.  Her  text 
and  rubric  results  demonstrate  a level  of  quality  possible  in  the  expanded  assessment 
process  the  Massachusetts  Curriculum  Frameworks  favors. 

In  her  assessment  process,  Reidester  examines  the  task,  the  student's  performance 
and  her  own  presentation  of  the  task  in  order  to  determine  hat  route  her  instruction 
will  follow  in  the  days/  weeks  ahead.  The  use  of  the  Student  Rubric  allows  her  to 
pinpoint  Esther's  strengths  and  weaknesses,  so  in  subsequent  lessons  these  can  be 
developed  and  reassessed  to  determine  both  Esther's  real  progress  and  the  efficacy  of 
lessons  presented  to  her.  The  Rubrics  instrument  eases  the  in-depth  analysis  process 
while  forcing  Reidester  (and  any  who  use  it)  to  establish  the  criteria  on  which  she  is 
basing  her  assessments.  Perhaps  most  importantly,  the  rubric  is  a concrete  report  on 
which  to  base  discussions  with  students  about  academic  progress  and  needs 

THE  PROBLEM  TASK: 

Using  the  Greenfield  Recorder  newspaper,  find  the  average  rent  for  a one  bedroom  and  a two 

bedroom  apartment.  Graph  your  results.  Answer  these  questions  using  the  graph. 

1 . Is  a two  bedroom  apartment  in  Greenfield  more  or  less  than  a two  bedroom 
apartment  in  Athol?  By  how  much? 

2.  Which  town  has  the  lowest  average  rent  for  a one  bedroom  apartment?  For  a two 
bedroom  apartment? 

3.  Where  can  one  find  the  highest  rent  for  a one  bedroom  apartment? 

4.  What  is  the  difference  between  the  cost  of  a one  bedroom  and  a two  bedroom 
apartment  in  Orange? 


by 

Peg  Reidester 


Franklin  Hampshire 
Employment  and 
Training  Consortium 
Athol,  MA 
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REIDESTER'S  ANALYSIS 

Context:  This  exercise  was  done  mainly 
to  reinforce  the  skill  of  averaging  numbers 
which  the  class  had  been  working  on  for 
the  previous  two  weeks.  It  was  also  used 
to  see  which  students  had  any  experience 
with  graphing  and  interpreting  graphs  ( 
sort  of  a pre  assessment).  In  addition,  it 
was  used  to  give  the  students  the  opportu- 
nity to  use  real  life  numbers  and  situations 
in  math  class. 

Outcome:  It  was  soon  obvious  to  me  that 
Esther  was  not  at  all  familiar  with  making 
graphs.  She  needed  a lot  of  assistance  and 
did  not  show  a sense  of  the  detail  required 
when  setting  up  a graph.  For  example,  the 
spacing  is  off:  The  bars  are  not  separated 
by  a fixed  number  of  units.  Also,  when 
labeling  her  axes,  she  wasn't  specific 
enough.  Someone  trying  to  interpret  the 
graph  would  have  some  difficulty.  I 
believe  these  problems  come  from  her  lack 


of  familiarity  with  graphing,  but  she  could 
have  a problem  with  arranging  things  in 
space,  as  well.  I would  want  to  work  with 
her  more  on  this  to  determine  which  it  is. 

Esther's  computation  skills  proved 
adequate  for  this  assignment.  She  was  able 
to  calculate  the  average  rents  and  subtract 
whole  numbers.  She  didn't  pass  in  a data 
sheet  with  her  graph,  though,  which  may 
indicate  a lack  of  awareness  of  the  connec- 
tion between  data  and  graphs. 

Conclusion:  I think  this  exercise  was  a 
good  assignment.  It  provided  me  with 
information  on  the  students'  abilities  to 
collect  and  organize  data,  compute  aver- 
ages, graphs  and  interpret  graphs.  Next 
time,  I wouldn't  use  it  until  I had  done 
some  practicing  with  graphing  beforehand 
because  I would  like  to  see  students 
perform  the  entire  exercise  as  much  on 
their  own  or  with  the  help  of  each  other 
as  possible. 
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STUDENT  ASSESSMENT  RUBRIC 

Name:  Esther 

Assessment  Task::Graphing  Average  Rents 
Date:  5/12/96 
Skills  Assessment: 

3 - mastery  2 - demonstrated  use  1 - unused  or  misused 

Skills  Assessed  in  Task 

1 . Computation  (subtracting  whole  numbers,  dividing) 

2.  Finding  the  average 

3.  Comparing  averages 

4.  Following  directions  for  setting  up  graphs 

5.  Using  a graph  to  answer  questions 

6.  Recording  data 

7.  

8.  

Partial  list  of  skill  types  to  be  assessed: 

computation — two-digit  multiplication,  etc.;  solving  equations; 
notation — student  use  and  student  comprehension; 
communication — vocabulary  comprehension  or  use; 

expression  of  probabilities;  coordinate  placement;  measurement  conversion; 
formula  use;  pattern  recognition  or  description;  finding  a percent- 
locating  numbers  on  a number  line;  simplifying  fractions... 


Competency  Level 

2 

2 

2 

2 

2 

3 
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STUDENT  ASSESSMENT  RUBRIC 

Name: 

Assessment  Task:: 

Date:  

Skills  Assessment: 

3 - mastery  2 - demonstrated  use  1 - unused  or  misused 

Skills  Assessed  in  Task  Competency  Level 

1 . Computation  {subtracting  whole  numbers,  dividing) 

2.  Finding  the  average 

3.  Comparing  averages 

4.  Following  directions  for  setting  up  graphs 

5.  Using  a graph  to  answer  questions 

6.  Recording  data 

7.  

8.  

Partial  list  of  skill  types  to  be  assessed: 

computation — two-digit  multiplication,  etc.;  solving  equations; 
notation — student  use  and  student  comprehension; 
communication — vocabulary  comprehension  or  use; 

expression  of  probabilities;  coordinate  placement;  measurement  conversion; 
formula  use;  pattern  recognition  or  description;  finding  a percent; 
locating  numbers  on  a number  line;  simplifying  fractions... 
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HABITS  OF  MIND  EXHIBITED 


3 - highly  visible  2 - evident  1 - not  evident  0 - N/A 

Affective  Domains  Assessed  in  Task  Expression  Level 

1 . Persistence  (sticks  with  problem)  2 

2.  Curiosity  (engages  in  problem)  2 

3.  Flexibility  (attempts  alternative  solution  methods)  0 

4.  Thoroughness  (checks  answers,  responds  to  all  questions,  2 

compiles  sufficient  data) 

5.  Creativity  (unique  approaches  responses  or  presentation)  0 

6.  Cooperation  (shares  ideas  and  materials,  listens)  2 

7.  Communication  (states  ideas  clearly,  asks  appropriate  questions)  1 

8.  Reasoning  (shows  logical  and/or  intuitive  reasoning;  inductive  1 

and/or  deductive  reasoning;  proportional;  generates  hypotheses) 

9.  Problem  solving  (uses  a variety  of  strategies  and/or  appropriate  1 

Strategy;  poses  interesting,  sensible  problems) 


Strengths/Difficulties  exhibited: 

* reading  comprehension  * choosing  strategy  * computational  accuracy 

* choosing  operation  * efficient  w/time  * procedural  knowledge 

* extending  or  applying  knowledge 

General  Comments: 

Esther  easily  recorded  the  data  on  rents  and  had  no  trouble  computing  the  average 
which  had  been  previously  been  new  to  her.  Obviously,  she  has  progressed  here, 
though  the  exercise  did  not  make  clear  whether  or  not  she  understands  the  distinctions 
between  mean  and  median.  She  needed  a lot  of  guidance  for  the  graph  which  indi- 
cates a lack  of  familiarity  with  graphing.  Her  spatial  sense  of  the  graph  was  weak. 

Recommendations: 


Overall  Assessment  (Math) 

1 - needs  prior  skills/concepts  2 - needs  practice  3 - ready  to  progress 
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HABITS  OF  MIND  EXHIBITED 

3 - highly  visible  2 - evident  1 - not  evident  0 - N/A 

Affective  Domains  Assessed  in  Task  Expression  Level 

1 . Persistence  (sticks  with  problem) 

2.  Curiosity  (engages  in  problem) 

3.  Flexibility  (attempts  alternative  solution  methods) 

4.  Thoroughness  (checks  answers,  responds  to  all  questions, 
compiles  sufficient  data) 

5.  Creativity  (unique  approaches  responses  or  presentation) 

6.  Cooperation  (shares  ideas  and  materials,  listens) 

7.  Communication  (states  ideas  clearly,  asks  appropriate  questions) 

8.  Reasoning  (shows  logical  and/or  intuitive  reasoning;  inductive 
and/or  deductive  reasoning;  proportional;  generates  hypotheses) 

9.  Problem  solving  (uses  a variety  of  strategies  and/or  appropriate 
Strategy;  poses  interesting,  sensible  problems) 

Strengths/ Difficulties  exhibited: 

□ reading  comprehension  □ choosing  strategy  □ computational  accuracy 

□ choosing  operation  □ efficient  w/time  □ procedural  knowledge 

□ extending  or  applying  knowledge 

General  Comments: 


Recommendations: 


Overall  Assessment  (Math) 

1 - needs  prior  skills/concepts  2 - needs  practice  3 - ready  to  progress 
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Assessment  Package  Offers  Useful  Resource 
for  Novice  and  Experienced  Practitioner  Alike 


Collaborations  Assessment  Package 
Level:  Beginning  1,  Beginning  2 
Heinle  and  Heinle  Publisher 
A Division  of  International  Thomson 
Publishing,  Inc. 

Boston,  MA  02116 
Author  Cathy  C.  Shank 


by 

Jeanne  Kearsely 


City  College  of 
San  Francisco 
San  Francisco,  CA 


Teachers  working  in  non-credit  or 
workplace  education  programs 
face  the  ongoing  challenge  of 
accounting  for  or  demonstrating  student 
progress.  Quantifying  student  progress  on 
a form  acceptable  to  funders  and  or  pro- 
gram administrators  can  be  challenging. 
Standardized  or  commercially  available 
tests,  which  produce  funder  friendly 
numbers,  are  inappropriate  to  most 
learner-centered,  non  academic,  adult 
education  programs.  Consequently, 
instructors  create  their  own  assessment 
tools  to  more  accurately  test  students  on 
the  content  and  instructional  methods 
utilized  by  their  programs.  The  difficulty 
of  translating  the  test  results  into  a more 
usable  form  for  the  number  of  audiences 
remains,  however.  The  assessment  pack- 
age from  the  Collaborations  series  presents 
a possible  solution  to  this  task. 

Collaborations  is  a five-year  series 
designed  for  the  non-credit,  adult  educa- 
tion program.  The  series  is  based  on  the 
tenets  of  a learner-centered  and  holistic 
approach  to  ESOL  instruction.  In  addition 
to  the  students  books',  workbooks,  teach- 
ers' manuals  and  supplemental  materials, 
each  level  also  includes  an  assessment 


package.  The  package  is  intended  to  be 
used  in  conjunction  with  the  Collabora- 
tions series.  Its  design  is  generic  enough, 
however,  to  be  used  independently  of  the 
series. 

The  materials  in  the  assessment  package 
do  not  break  any  new  ground  in  the  area 
of  authentic,  learner-centered  assessment. 
The  majority  of  its  instruments  and  assess- 
ment procedures  are  fairly  traditional,  like 
the  initial  intake  interview  and  the  cloze 
passages,  and  have  been  used  in  adult 
education  programs  for  many  years.  The 
author  does  highlight  the  use  of  portfolio 
assessment  as  a separate  alternative 
section. 

The  material  is  clearly  presented  and  the 
tools  can  be  adapted  by  an  entire  program 
or  as  needed  by  classroom  instructors. 
Flexibility  in  a commercially-prepared 
assessment  instrument  is  extremely 
important  and  rare.  The  package  covers  all 
aspects  of  the  student  assessment  process: 
initial,  ongoing,  and  mid-  and  end-of-level 
assessment. 

Examples  of  self  assessment  techniques 
and  course  evaluation  forms  are  also 
included  in  the  package.  The  content  of 
the  forms  can  be  readily  changed  to  reflect 
the  material  covered  by  a specific  class  or 
program.  I particularly  liked  the  author's 
concrete  suggestions  for  incorporating 
various  assessment  practices  into  everyday 
classroom  routine,  for  example  pair  work, 
interviewing  and  reporting,  and  informa- 
tion gap  exercises.  These  are  all  valid 
forms  of  ongoing  assessment. 


•VENTURES  IN  ASSESSMENT 


Volume  9:  December  1996 


Volume  9 • 55 


While  students  are  engaged  in  these 
activities  the  teacher  is  free  to  circulate  and 
evaluate  students.  The  author  also  makes  a 
point  of  identifying  activities  that  teachers 
may  already  use  in  the  classroom  and 
suggest  ways  of  incorporating  them  into 
the  assessment  process.  I think  it  was 
brilliant  and  I wish  she  had  developed 
the  point  further. 

Another  strength  of  the  assessment 
package  is  that  it  is  extremely  comprehen- 
sive. Charts  throughout  the  package 
clearly  illustrate  the  content,  uses  and 
skills  tested  by  the  various  instruments. 
The  forms,  scripts  and  scoring  keys  are 
clear  and  comprehensible.  These  forms  are 
duplication  ready  and  accompanied  by 
very  explicit  instructions.  Clear  and  basic 
instructions  are  essential  for  practitioners 
with  limited  experience  implementing 
non-academic  assessment  tools.  Some  of 
the  author's  advice  may  seem  too  basic  — 
"remember  to  smile"  and  "make  students 
feel  welcome  and  comfortable."  Some- 
times, though,  these  small  details  that  we 
assume  everyone  knows  need  to  be  explic- 
itly stated  for  the  benefit  of  newcomers 
and  experienced  practitioners  alike. 

In  many  ways  the  assessment  package 
is  similar  to  a survey  course  of  "learner- 
friendly"  assessment  practices,  theories 
and  techniques.  I particularly  like  that  the 
author  included  a brief  synopsis  of  the 


theory  behind  many  of  the  practices  she 
utilized.  A clear  correlation  between 
theory  and  application  is  particularly 
helpful  to  those  new  in  the  field.  Unfortu- 
nately, as  happens  with  most  survey 
courses,  the  treatment  of  the  subject  matter 
can  at  times  seem  somewhat  superficial 
and  overly  simplistic. 

I found  this  to  be  particularly  true  when 
the  author  gives  advice  for  orienting  low- 
level  students  to  less  traditional  initial 
intake  and  ongoing  assessment  proce- 
dures. For  example,  suggestions  such  as 
"use  facial  expressions,  body  language 
and  demonstrations"  are  not  particularly 
helpful  when  a teacher  is  trying  to  convey 
the  potentially  confusing  concepts  in- 
volved in  portfolio  assessment.  Teachers 
working  with  very  low-level  students  who 
want  to  use  alternative  methods  of  assess- 
ment need  more  practical  and  concrete 
advice. 

Overall,  the  Collaborations  Assessment 
Package  is  a very  useful  resource  for 
classroom  teachers  and  program  coordina- 
tors. I think  the  assessment  package  would 
be  particularly  helpful  to  individuals  who 
have  limited  experience  creating  or  imple- 
menting assessment  tools  in  non-  academic 
or  workplace  learning  situation.  It  is  a well 
presented  collection  of  assessment  tools 
easily  accessible  and  implemented  by 
either  experienced  or  novice  practitioners. 
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S A B E S 

System  for  Adul  Baste  Education  Support 

m. 

SABES  is  the  System  for  Adult  Basic  Education  Support,  a comprehensive  training  and 
technical  assistance  initiative  for  adult  literacy  educators  and  programs.  Its  goal  is  to 
strengthen  and  enhance  literacy  services  and  thus  to  enable  adult  learners  to  attain  literacy 
skills. 

SABES  accomplishes  this  goal  through  staff  and  program  development  workshops,  consulta- 
tion, mini-courses,  mentoring  and  peer  coaching,  and  other  training  activities  provided  by 
five  Regional  Support  Centers  located  at  community  colleges  throughout  Massachusetts. 
SABES  also  offers  a 15-hour  Orientation  that  introduces  new  staff  to  adult  education  theory 
and  practice  and  enables  them  to  build  support  networks. 

SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect,  evaluate,  and  disseminate 
ABE  materials,  curricula,  methodologies,  and  program  models,  and  encourages  the  develop- 
ment and  use  of  practitioner  and  learner-generated  materials.  Each  of  the  five  SABES 
Regional  Support  Centers  similarly  offers  program  support  and  a lending  library.  SABES 
maintains  an  Adult  Literacy  Hotline,  a statewide  referral  service  which  responds  to  calls 
from  new  learners  and  volunteers.  The  Hotline  number  is  1-800-447-8844. 

BT 

The  SABES  Central  Resource  Center , a program  of  World  Education , publishes  a statewide 
quarterly  newsletter , " Bright  Ideas/'  and  journals  on  topics  of  interest  to  adult  literacy 
professionals , such  as  this  volume  of  " Adventures  in  Assessment " 

The  first  three  volumes  of  " Adventures  in  Assessment"  present  a comprehensive  view  of  the 
state  of  practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners. 
Volume  1,  Getting  Started,  includes  start-up  and  intake  activities ; Volume  2,  Ongoing, 
shares  tools  for  ongoing  assessment  as  part  of  the  learning  process;  Volume  3,  Looking 
Back,  Starting  Again,  focuses  on  tools  and  procedures  used  at  the  end  of  a cycle  or  term, 
including  self,  class,  and  group  evaluation  by  both  teachers  and  learners.  Volume  4 covered 
a range  of  interests,  and  Volume  5,  The  Tale  of  the  Tools  is  dedicated  to  reflecting  on 
Component  3 tools  of  alternative  assessment.  Volume  6,  Responding  to  the  Dream  Con- 
ference, is  dedicated  to  responses  to  Volumes  T5.  Volume  7,  The  Partnership  Project, 
highlighted  writings  from  a mentoring  project  for  practioners  interested  in  learning  about 
participatory  assessment.  Volume  8 covered  a range  of  topics  from  education  reform  to 
learner  involvement  in  assessment.  Volume  9 looked  at  assessment  in  a volunteer  program, 
native  language  literacy,  ESOL  programs,  and  evaluations  in  workplace  education  programs. 

In  this  volume,  many  different  practitioners  offer  their  view  on  alternative  assessment  and 
the  BEST. 
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We'd  like  to  see  your  contribution . Contact  Editor  Alison  Simmons  to  discuss  your  submission. 

Opinions  expressed  in  "Adventures  in  Assessment"  are  those  of  the  authors  and  not  necessarily 
the  opinions  of  SABES  or  its  funders. 

Permission  is  granted  to  reproduce  portions  of  this  journal ; we  request,  however,  appropriate 
credit  be  given  to  the  author  and  to  Adventures  in  Assessment. 

Adventures  in  Assessment  is  free  to  DOE-funded  Massachusetts  programs ; out-of-state 
requests  will  be  charged  a nominal  fee.  Please  write  to,  or  call: 

Alison  Simmons 
World  Education 
44  Farnsworth  Street 
Boston,  MA  02210-1211 
617-482-9485 
asimmons@JSI.com 
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Introduction 

Volume  10:  Time  to  Reflect 


As  the  new  year  approaches  we 
are  filled  with  anticipation  about 
what  lies  ahead  for  Adventures 
in  Assessment  and  alternative 
assessment.  Since  this  is  the  tenth  volume, 

I guess  I am  feeling  compelled  to  reflect  a 
bit  and  revisit  the  purposes  for  Adventures. 
In  Volume  #6,  Loren  McGrail  talked  about 
the  origins  of  the  journal  and  the  purpose 
and  audience  she  had  in  mind  when  she 
started  it  in  1991.  She  wrote: 

"...  in  addition  to  providing  a framework 
and  a forum  for  alternative  assessment,  I saw 
the  creation  of  a field-based  journal  as  a golden 
opportunity  to  put  into  practice  a process 
approach  to  writing,  a way  for  practitioners  to 
experience  first-hand  the  power  of  having  their 
writing  responded  to  with  non-evaluative  feed- 
back. . . All  authors  commented  on  the  self 
knowledge  they  gained  by  the  act  of  writing 
itself.  I had  underestimated  the  power  of  writ- 
ing to  reflect  back  to  us  what  we  really  think 
and  believe.  And. . .1  was  surprised  to  hear, 
repeatedly,  the  desire  many  authors  expressed 
to  connect  with  others  so  they  could  get  feed- 
back on  their  own  practice.  ...lam  also  struck 
by  some  authors'  clarity  about  the  need  to  get 
this  information  out. . . "I  wanted  to  support 
the  cause  for  alternative  assessment  and 
remind  people  that  there  are  other  options 

to  the  T ABE." 

Adventures  in  Assessment  was  always  in- 
tended as  a staff  development  journal  for 
authors  reflecting  on  their  own  experience 
and  sharing  with  others,  as  well  as  for 
readers  interested  in  looking  at  other  ways 
to  do  assessment  in  their  programs  and 
classrooms.  When  I am  on  my  annual  jour- 
ney in  search  of  adventurers  to  write  about 
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their  practice,  I am  struck  by  the  number 
of  practitioners  who  feel  they  have  little  to 
offer  others  in  the  area  of  assessment.  Al- 
though, I hear  about  a lot  of  great  ideas, 
tools  and  questions,  it  is  hard  to  convince 
people  that  they  have  a lot  to  offer  and  can 
write  about  a process,  an  idea,  or  a ques- 
tion without  having  the  ultimate  answer. 

For  those  who  do  not  know  already, 
there  is  not  a clear  bright  light  at  the  end  of 
the  assessment  tunnel  that  will  resolve  all 
the  many  dimensions  of  assessment  that 
are  inherent  in  our  classrooms  and  pro- 
grams. What  each  teacher /practitioner  can 
offer  is  their  take  on  assessment  and  the 
ways  in  which  they  come  to  terms  with  the 
many  challenges  assessment  poses.  This 
may  take  the  form  of  tools,  ideas,  ques- 
tions or  a review  of  other  materials.  It  is 
these  ideas  and  this  format  that  make 
Adventures  in  Assessment  unique  among 
the  journals  and  books  on  assessment.  I 
feel  we  have  continued  in  the  spirit  with 
which  Loren  began  this  journal  and  hope 
we  do  not  to  lose  sight  of  Adventures' 
importance  as  a field-based  journal  for 
teachers  and  practitioners  in  search  of 
an  adventure  in  assessment. 

What  might  the  future  hold  for  Adven- 
tures in  Assessment ? As  the  standards- 
based  initiatives  take  hold  both  nationally 
and  statewide  (Equipped  for  the  Future, 
Curriculum  Frameworks,  Skill  Standards 
for  Workplaces,  etc.) , it  will  be  interesting 
to  watch  how  the  field  responds  to  these 
initiatives  and  how  it  looks  at  and  defines 
assessment.  Will  these  initiatives  help  us 
develop  a common  language  and  common 
expectations  for  outcomes?  Will  we  align 
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our  instruction  and  assessment  practices 
with  the  standards?  Adventures  in  Assess- 
ment seems  more  relevant  now  as  we  enter 
a time  of  content  standards  and  outcomes 
from  our  field,  as  well  as  other  fields  that 
define  what  our  learners  should  know  and 
be  able  to  do.  Adventures  will  continue  to 
offer  a forum  where  practitioners  can 
document  their  practices  and  we  will  see 
where  these  experiences  take  us. 

This  tenth  volume  of  Adventures  in  As- 
sessment has  a great  assortment  of  writers 
looking  for  ways  to  integrate  assessment 
into  their  current  teaching/learning. 

Marta  Mangan-Lev  writes  about 
authentic  assessment  and  cooperative 
learning.  We  look  at  how  she  applies  the 
principles  of  one  educational  theory  to  her 
classroom  and  the  principles  of  authentic 
assessment.  She  believes  that  our  assump- 
tions about  teaching  and  learning  should 
integrate  our  instructional  methods  with 
our  assessment  practices. 

Maria  Kephallenou  talks  about  her 
experience  at  the  Haitian  Multi-Service 
Center  in  Boston.  She  looks  at  questions 
and  issues  that  constantly  surface  in  her 
program  around  assessment,  the  complex 
nature  of  assessment  in  a multi-service 
center,  and  her  view  of  assessment  as  she 
moved  from  teacher  to  administrator. 

Judy  Chau  asks  us  to  think  about  what 
we  assess  and  whether  we  are  too  focused 
on  skill  areas  and  spend  less  time  with 
fewer,  less  tangible  skill  areas.  She  offers 
us  a peer  evaluation  tool  for  interviewing 
that  focuses  on  those  less  tangible  areas. 

Martha  Jean  takes  a look  back  at  her 
EGAP  assessment  tool  that  has  been  used 
and  adapted  by  other  teachers  in  the  field. 
She  talks  about  the  minor  adjustments 
made  partly  due  to  her  work  in  Learning 
Disabilities  and  Multiple  Intelligences. 


Sylvia  Greene,  Nancy  Hoe  and  Lally 
Stowell  take  us  on  a trip  through  the  as- 
sessment process  in  their  family  literacy 
program.  What  seems  like  miles  and  miles 
of  assessment  protocol  is  really  a very 
comprehensive  system  that  aims  to  look  at 
the  whole  learner  and  cover  all  the  areas 
where  they  are  making  progress.  What 
struck  me  about  this  piece  was  that  all  of 
the  information  gathered  near  the  begin- 
ning of  the  learners'  time  at  the  Center  is 
used  to  help  students  identify  their  goals 
for  learning.  Their  initial  assessment 
reveals  areas  of  strengths  and  weaknesses, 
then  the  information  is  used  to  help  stu- 
dents write  goals  and  objectives. 

The  Operation  Bootstrap  Health  Team 
assesses  community  health  needs  at  an 
adult  education  program  in  Lynn,  MA. 
After  an  initial  assessment,  the  team  devel- 
ops a program  for  the  students  centered 
around  a topic.  Along  the  way,  students 
assess  the  effectiveness  of  their  program 
as  well  as  how  they  are  working  within 
the  team. 

Kathy  Sikes  (interviewed  by  Melody 
Schneider)  talks  about  her  experience 
implementing  and  training  volunteers 
in  the  use  of  portfolio  assessment. 

Beth  Bingman  from  the  University 
of  Tennessee  gives  us  an  update  on  the 
National  Center  for  the  Study  of  Adult 
Learning  and  Literacy's  (NCSALL)  assess- 
ment/outcomes research. 

In  Voices  From  the  Field  practitioners 
look  at  the  Basic  English  Skills  Test  (BEST). 
Moira  Lucey  gives  us  a history  of  the  BEST 
and  valuable  information  about  its  design 
and  purpose.  Barbara  Lippell-Paul  looks 
at  the  BEST  from  an  historical  perspective 
and  helps  us  to  look  critically  at  what  the 
test  is  asking  us  to  do  and  what  some  of 
the  issues  are  with  administering  the  test. 
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Dulany  Alexander  looks  at  the  BEST  by 
comparing  the  profiles  of  two  ESOL  stu- 
dents. Rachel  Donnelly,  a VISTA  volun- 
teer, relates  some  of  the  insights  she  has 
had  as  a "beginner"  with  the  BEST. 

Cathy  Coleman  in  Learning  from  Expe- 
rience reflects  on  the  use  of  the  TABE  (Test 
of  Adult  Basic  Education)  in  her  program. 
She  suggests  questions  to  consider  when 
using  a standardized  test  but  mainly  to 
question  if  the  test  is  a good  fit  with  what 
your  learners  and  program  want  to  accom- 
plish. 

In  What  Counts?,  Ken  Tamarkin  offers 
a way  to  look  at  assessment  in  a computer 
class  at  Malden  Mills.  He  looks  at  tools  he 
uses  for  placement,  progress,  and  program 
evaluation.  He  involves  students  from  the 
beginning  in  developing  and  understand- 
ing the  assessment  process. 


Finally,  Caroline  Gear  looks  at  the  book 
Phenomenal  Changes:  Stories  of  Participants 
in  the  Portfolio  Project  as  a staff  develop- 
ment tool.  In  the  first  of  two  articles,  she 
shares  with  us  how  she  and  her  staff  are 
using  the  book  to  help  them  in  their  work 
on  portfolio  assessment. 

It  is  a full  volume  of  tools  and  ideas. 

As  always  we  welcome  your  comments 
and  suggestions.  If  you  wish  to  submit  an 
article  or  respond  to  an  article  in  this  issue, 
feel  free  to  contact  me  at  the  address  below. 
The  authors  would  appreciate  your  feed- 
back and  ideas. 


Alison  Simmons 
World  Education/SABES 
44  Farnsworth  Street 
Boston,  MA  02210 
asimmons@jsi.com 
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The  Connection  between  Cooperative 
Learning  and  Authentic  Assessment 


Assessment.  Evaluation.  The  words 
/%  conjure  images  of  tests  — sharp  #2 
X A pencils,  rows  of  bubbles  to  be 
darkened  in,  quiet  rooms  with  perspiring 
test-takers  and  only  the  sound  of  rustling 
paper.  I'm  learning,  along  with  my  learn- 
ers, to  replace  these  dated  images  with  a 
reality  that  conforms  more  with  my  values 
about  teaching  and  learning.  We're  learn- 
ing to  use  authentic,  learner-centered  tools 
which  integrate  assessment  into  the  cycle 
of  teaching/leaming.  We're  learning  to  use 
assessment  as  a tool  for  self  understanding 
and  instructional  planning. 

There  are  other,  equally-dated  images  I 
can  conjure  from  my  educational  experi- 
ences: straight  rows  of  wooden  desks,  one 
behind  the  other;  a teacher's  desk  facing 
the  rows  of  students;  a teacher,  talking, 
talking,  and  writing  on  the  broad  expanse 
of  a blackboard,  occasionally  calling  upon 
a student  to  supply  an  answer,  with  luck 
the  correct  one. 

My  classroom  doesn't  look  like  this. 

We  start  out  seated  around  a horseshoe  of 
tables,  each  learner  facing  the  others.  We 
often  work  in  cooperative  groups,  for 
which  learners  pull  chairs  into  small  clus- 
ters to  work  together.  My  position  is  not 
at  the  front  of  the  room  lecturing  (well, 
maybe  once  in  a while),  but  usually  circu- 
lating among  learners  engaged  in  using 
language,  engaged  in  learning.  We're 
learning  to  create  and  work  in  a learning 
environment  in  which  learners  work  to- 
gether cooperatively  to  achieve  shared 
goals. 
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Cooperative  learning  in  the  adult  edu- 
cation classroom  offers  some  valuable  op- 
portunities for  authentic  assessment.  In 
my  work  with  cooperative  groups  and  au- 
thentic assessment  I have  learned  there  are 
many  parallels  between  the  two.  This  has 
made  it  easier  to  integrate  them.  To  see 
this  more  clearly  we  will  first  look  at  some 
principles  of  cooperative  learning. 

THE  STRUCTURE  OF 
COOPERATIVE  LEARNING 

If  you  were  to  survey  teachers  of  adult 
education  about  their  use  of  groupwork, 
nearly  all  would  probably  respond  that 
they  often  have  people  working  in  groups. 
In  my  ESOL  classes  learners  may  work  in 
groups  formed  using  a number  of  random 
techniques:  counting  off  by  3s  or  4s;  dis- 
tributing three  or  four  different  pictures 
and  then  forming  a group  with  those  who 
have  the  same  picture,  choosing  a topic 
(for  example,  a kind  of  music  or  food)  and 
then  grouping  those  who  made  the  same 
choice.  While  learners  work  together  in 
these  groups  for  conversation  or  to  com- 
plete a task,  they  are  not  explicitly  coop- 
erative. Cooperative  groupwork  is  distinct 
from  work  in  small  groups: 

• learners  work  in  positive  interdependence, 

• each  participant  has  a clear  and  specific 
role, 

• the  process  of  working  together  is  an 
important  focus, 

• participants  reflect  upon  and  analyze 
their  work  together. 


by 

Martha  Mangan  Lev 


Valley  Opportunity 
Council 
Chicopee,  MA 
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A cooperative  model  of  groupwork  is 
structured  in  these  specific  ways.  The  role 
of  the  teacher  is  to  set  up  those  structures 
by  considering  these  questions:  What  will 
learners  do?  What  role  will  each  learner 
play?  How  will  they  reflect  upon  their 
work  together?  Equally  important,  the 
teacher  considers  the  personality,  skills, 
and  learning  style  of  each  learner  to  form 
groups  that  can  work  together  effectively. 
Next,  she/he  facilitates  the  formation  of 
the  group  and  observes  them  in  action. 
Finally,  the  teacher  provides  learners  with 
a means  to  reflect  upon  and  evaluate  their 
group's  work;  that  is,  she  incorporates 
assessment  tools. 

About  now,  you  may  be  thinking, 

"Yikes,  I've  got  30  minutes  to  plan 
tomorrow's  class.  Let's  just  count  off  by  3s 
and  talk  about  the  weekend.  We  can  work 
cooperatively  sometime  when  I've  got  a 
day  or  two  to  plan!" 

It's  true  that  the  most  extensive  work  of 
the  teacher  is  in  planning  cooperative  in- 
teractions, in  setting  up  just  those  condi- 
tions outlined  above.  It's  equally  true  that 
there's  a learning  curve  in  developing  and 
honing  our  skills  as  facilitators  of  coopera- 
tive learning;  the  initial  time  investment  is 
significant.  As  we  become  more  adept  at 
the  process,  it  takes  less  time. 

We  find  the  same  is  true  of  using  au- 
thentic assessment.  We  can  opt  to  use 
'quick  and  dirt/  standardized  tests,  or  can 
invest  the  time  to  develop  authentic  assess- 
ment tools  that  also  collect  information 
about  learners'  skills,  learning  styles  and 
personalities. 

The  payoff  for  investing  in  these  two 
approaches  is  significant.  Teaching  and 
learning  are  enhanced  as  learners  acquire 
the  habits  of  reflecting  upon  their  learning 
and  working  with  others.  Our  teaching,  by 
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integrating  on-going  feedback  from  learn- 
ers, also  becomes  more  relevant  and  mean- 
ingful in  learners'  lives. 

One  way  to  simplify  the  planning  of 
cooperative  groupwork  is  to  think  of  it 
as  three  sets  of  variables  to  be  sorted  and 
matched:  1)  the  content  and  process  of  the 
task  to  be  carried  out,  2)  the  attributes  of 
the  learners  to  be  grouped  together,  and 
3)  the  assessment  component:  the  tools 
learners  will  use  to  reflect  upon  their 
experience. 

Content/process  of  the  task 

This  is  a starting  point  for  any  lesson 
planning.  What  exactly  will  learners  do 
together?  What  skills  will  they  learn/ 
practice?  What  are  the  goals  of  their  work? 
What  roles  can  group  members  fill  in  order 
to  achieve  the  goals?  With  the  possible  ex- 
ception of  the  last,  these  questions  underlie 
any  lesson  planning.  In  this  sense,  the 
planning  is  the  same,  although  the  task 
will  be  tailored  to  the  structure  of  a group 
working  cooperatively.  While  appropriate 
roles  will  vary  according  to  the  nature  of 
the  activity,  there  are  a number  of  fre- 
quently relevant  roles  which  can  also  be 
customized  to  the  particular  activity.  The 
level  of  support  for  each  role — checklists 
or  other  ways  to  clearly  specify  what  the 
learner  will  do  in  that  role — can  be 
adapted  to  suit  the  level(s)  of  the  learners. 

Possible  roles  to  assign  may  include  an 
observer,  a questioner,  a timekeeper,  and  a 
summarizer  and  facilitator. 

Observer,  watches  the  work  of  the  group, 
often  equipped  with  a checklist  of  spe- 
cific behaviors  to  look  for  (e.g.,  does 
everyone  speak?  Are  all  ideas  treated 
with  respect?). 

Questioner:  asks  questions  of  one  or  more 
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participants.  This  learner  could  also  be 
provided  a list  (e.g.,  of  question  words 
[who,  what,  etc.],  or  of  specific  questions). 

Timekeeper:  keeps  the  group  within  time 
limits  set  or  agreed  upon. 

Summarizer:  may  sum  up  the  work  of  the 
group  herself,  or  may  present  a group- 
developed  summary  to  the  class. 

Facilitator:  helps  the  group  accomplish  its 
agreed-upon  tasks. 

Grouping  Learners:  Attributes  of 
Learners  to  be  Grouped  Together 
The  next  piece  of  the  puzzle  is  learner 
attributes.  As  I get  to  know  my  learners, 

I pay  attention  to  their  individual  differ- 
ences in  an  on-going  process  of  assessment 
through  observation.  I use  this  information 
to  mix  and  match  qualities  to  increase  a 
group's  success.  Learner  Skills  are  one  set 
of  attributes:  language,  leadership,  facilita- 
tion, etc.  Another  variable  is  the  consis- 
tency of  attendance,  which  will  be 
particularly  relevant  for  groups  working 
on  a project  over  time. 

Native  language  should  be  taken  into 
consideration:  would  the  task  be  best  ac- 
complished in  a group  as  heterogeneous  in 
language  as  possible,  such  that  more  inter- 
action would  take  place  in  the  target  lan- 
guage? Or  would  learners  benefit  from  the 
ability  to  perform  parts  of  their  task  in  a 
shared  native  language? 

Learning  style  in  general  will  be  an  im- 
portant factor  in  group  interactions,  and 
particularly  an  individual's  preference  for 
working  independently  or  with  others.  Fi- 
nally, but  not  least  importantly,  the  mix  of 
personalities  in  a group  will  affect  its  suc- 
cess. A group  in  which  each  member  tends 
to  be  quiet  and  shy  may  have  trouble  get- 
ting going. 


Vignette: 

A group  from  my  Level  One  class  is  a won- 
derfully-varied and  complex  group  of  learners. 
Su,from  Korea,  has  very  good  listening,  speak- 
ing, and  reading  skills,  though  her  pronuncia- 
tion is  sometimes  difficult  for  others  to 
understand.  She  engages  enthusiastically  in  all 
tasks  and  attempts  to  involve  others.  Panay, 
from  Laos,  is  very  quiet  and  reluctant  to  speak. 
She  has  a good  sense  of  humor  and  understands 
most  of  the  classroom  talk.  Jay,  from  the  Philip- 
pines, is  the  youngest  member  of  the  class.  He 
also  has  very  good  language  skills,  but  is  easily 
distracted  from  a task.  The  fourth  member  of 
the  group,  Dorota,  a very  new  speaker  of  En- 
glish, tends  to  be  quiet.  She  has  little  confidence 
in  her  skills  but  is  determined  to  learn  English. 

The  configuration  of  this  group  meant  that 
English  is  their  only  common  language  and 
they  sometimes  struggle  to  communicate. 

Su  often  takes  leadership  in  getting  their  work 
started,  but  Dorota  also  pays  attention  to  keep- 
ing them  on  task.  Panay  sometimes  shyly  teases 
Jay,  who  enjoys  the  playful  interactions.  The 
group  works  well  together. 

AUTHENTIC  ASSESSMENT  IN 
COOPERATIVE  GROUP  WORK 

So,  okay,  you've  worked  out  some 
group  tasks,  figured  out  roles  that  you 
hope  will  work  to  accomplish  them,  and 
made  some  initial  groupings  of  learners. 
What  about  assessment?  What  information 
have  you  already  gathered?  What  will  you 
now  assess?  How? 

Through  the  structure  and  process  of 
cooperative  learning  learners  use  a wide 
range  of  skills.  These  skills  include: 

• language/content  skills  used  to  accom- 
plish the  group's  goals,  both  in  what 
students  do  and  how  they  do  it, 

• taking/following  leadership,  to  partici- 
pate in  any  of  the  various  roles,  and  to  to- 
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information  to 
mix  and  match 
qualities  to 
increase  a group's 
success. 
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gether  move  a group  to  achieve  its  goals, 

• negotiating  with  each  other  when  dif- 
ferent ideas  are  being  considered, 

• problem  solving:  clarifying  ideas,  elabo- 
rating the  ideas  suggested  by  others,  or 
seeing  the  consequences  of  particular 
solutions, 

• reaching  consensus,  a specific  and  not 
widely  familiar  decision-making  process 
that  honors  the  opinions  of  all  involved 
to  come  to  an  agreed-upon  outcome, 

• synthesizing/summarizing  in  order  to 
present  the  group's  work,  or  to  facilitate 
its  on-going  work, 

• observation/analysis,  identifying  what 
is  to  be  observed  and  how  to  under- 
stand what  one  is  seeing,  and 

• giving  feedback  to  other  learners  or  to 
the  instructor  about  the  group's  process, 
the  task,  and  other  aspects  of  the 
groupwork. 

Not  only  are  these  skills  critical  to  effec- 
tive groupwork,  they  will  also  enhance 
success  beyond  the  classroom,  in  the  work- 
place, and  in  the  community.  Those  skills 
more  related  to  process,  to  how  we  work 
with  others,  are  seldom  explicitly  elicited 
in  classroom  work  and  are  even  less  often 
evaluated  by  'traditional'  assessment.  By 
developing  tools  to  use  in  the  cooperative 
classroom,  learners  can  get  feedback  on 
their  skills  in  these  areas  as  well  as  develop 
them. 

Authentic  Assessment  Tools 

So  you've  got  some  groups  that  are, 
with  support  from  you,  working  together 
well.  You're  helping  learners  identify  and 
develop  skills,  such  as  those  listed  above, 
in  the  course  of  their  cooperative  efforts. 
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What  tools  can  you  and  they  use  to  reflect 
upon  and  evaluate  their  work?  Here  are 
some  suggestions. 

• learner  questionnaires  (see  box).  In  their 
simplest  form,  these  ask  learners  to 
choose  a response.  For  more  advanced 
learners,  questionnaires  may  also  ask  for 
more  extensive  responses. 


How  did  your  group  work  together  tonight? 

Name:  ’ Date: 

What  did  your  group  do  today? 


Circle  one 

I feel  good  not  good  about  my  group  today. 

I talked  a lot  a little  in  my  group. 

Other  people  talked  a lot  a little  in  my  group. 
My  group  helped  me  learn  a lot  a little  today. 
My  group  got  a lot  a little  work  done  today. 


• teacher  or  learner  observation.  The  ob- 
server may  use  a checklist  and  record 
the  frequency  of  particular  events  or 
may  watch  for  specific  behaviors. 

• checklists.  Completed  by  teacher  and/ 
or  learners,  they  may  include  specific 
content  skills  in  the  task  or  specific 
cooperative  skills. 

• reports/presentations  to  the  class.  In 
a variety  of  forms  (charts,  skits,  talks, 
etc.),  these  provide  a concrete  work 
product. 

• evaluation  of  groupwork  products. 
What  did  the  group  make/ present/ etc.? 
How  does  it  compare  with  their  goals? 

• learning  contracts.  These  provide  goal- 
based  evaluation  of  personal,  group,  or 
content  goals. 
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A variety  of  these  tools  can  and  should 
be  used  over  time.  Different  tools  will  ap- 
peal to  different  learners  and  elicit  a range 
of  perspectives  on  the  process  and/ or  the 
product  of  the  work.  This  assessment 
should  include  both  learner  and  teacher  in- 
put. A growing  desire  for  me  is  to  support 
learners  in  developing  tools  for  reflection 
and  evaluation  of  groupwork. 

As  is  usually  true  of  authentic  assess- 
ment, what  teachers  and  learners  learn 
from  these  tools  can  be  translated  into  con- 
tent for  subsequent  groupwork.  If  an  issue 
is  identified  as  a particular  strength  or 
weakness,  groupwork  can  be  designed  to 
address  this.  For  example,  if  one  of  the 
roles  is  that  of  'summarizer'  and  groups 
report  difficulty  in  carrying  this  out,  class 
instruction  can  increase  learner  under- 
standing and  skill  in  subsequent 
groupwork.  As  teachers  and  learners  be- 
come more  adept  in  the  process,  the  cycle 


of  groupwork  to  reflection  to  instruction 
to  groupwork  becomes  increasingly  mean- 
ingful and  on  target. 

Cooperative  groupwork  provides  an  op- 
portunity for  learners  to  express  and  build 
a range  of  social  and  intellectual  skills.  The 
principles  of  authentic  assessment  — 

• that  it  be  learner-centered  and  help 
learners  achieve  their  goals, 

• that  it  be  part  of  the  learning  experience, 

• that  it  use  a variety  of  procedures, 

• that  it  provide  feedback  that  will  lead  to 
better  instruction 

— are  consistent  with  those  guiding  coop- 
erative learning  and  make  it  the  appropri- 
ate technique  for  reflecting  upon  and 
evaluating  this  process.  Together,  coopera- 
tive learning  and  authentic  assessment  are 
powerful  tools  for  understanding  our- 
selves as  learners  and  as  teachers. 

ST 
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Assessment  in  ESOL 

The  Haitian  Multi-Service  Center  Experience 


by 


Maria 

Kephallenou 


Haitian  Multi-Service 
Center 

Dorchester,  MA 


The  Adult  Education  Program  at  the 
Haitian  Multi-Service  Center 
(HMSC)  has  developed  a program- 
wide assessment  process  that  responds  to 
the  needs  and  demands  of  our  program. 

As  part  of  the  on-going  development  of 
this  process  we  have  found  that  assess- 
ment is  an  important  programmatic  issue 
that  continually  needs  to  be  addressed.  As 
we  develop  and  refine  our  process,  we  ask 
ourselves  numerous  questions  and  address 
many  issues  about  our  processes  and  as- 
sessment in  general.  We  always  face  new 
challenges  and  new  perspectives  on  how 
to  use  assessment  more  efficiently. 

Some  issues  arise  time  and  time  again. 

In  a class  of  15  students  with  varying  edu- 
cational backgrounds,  individual  needs, 
ages,  priorities  and  language  skills,  for  ex- 
ample, assessment  of  progress  is  difficult. 
Students  do  not  progress  at  the  same  rate 
and  the  same  time.  Also,  individual  skills 
do  not  progress  at  the  same  rate.  How  do 
we  account  for  all  these  different  skills  and 
factors  and  assess  progress?  Most  likely, 
teachers  begin  by  focusing  on  the  skills: 
they  compare  a starting  point  — the  begin- 
ning of  the  class  — with  the  point  of  time 
of  the  final  assessment.  The  different  fac- 
tors (backgrounds,  schooling,  age,  etc.) 
are  then  taken  into  account  to  better  under- 
stand and  explain  why  progress  was /was 
not  made. 

Another  issue  deals  with  the  movement 
of  students  from  one  level  to  the  next.  Gen- 
erally, a student  is  ready  to  move  up  when 
an  overall  better  understanding  and  use  of 
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the  language  is  demonstrated.  It  includes 
an  initial  assessment  for  the  student,  a 
point  of  time  later  with  another  assessment 
and  a comparison  between  these  two. 
However,  "an  overall  better  understanding 
and  use  of  the  language"  is  very  vague. 

For  example,  how  does  one  assess  it?  Is  it 
the  same  for  all  students  or  are  there  varia- 
tions? If  so,  how  does  one  account  for 
them?  And  how  does  it  tie  to  the  progress 
of  individual  language  skills?  Because 
teachers  think  differently,  it  is  amazing 
that  most  students  move  with  few  problems. 

When  we  talk  about  student  progress  at 
our  meetings,  the  discussion  is  dynamic 
with  contributions  that  generate  plans  to 
address  issues.  The  discussion  becomes 
difficult,  however,  when  we  explore  ways 
to  capture  all  of  these  dynamics  in  a form 
that  is  simple  and  easy  for  students  to  un- 
derstand and  for  other  teachers  to  translate. 

Resolving  this  issue  depends  largely  on 
one's  position  in  a program,  the  purpose 
and  need  for  assessment,  and  how  it  fits 
into  the  whole  program.  Everybody  agrees 
on  the  importance  of  assessment  for  the 
students,  the  teachers,  and  the  program, 
but  we  all  have  different  perspectives  on 
its  purpose. 

A Student's  Perspective.  Students  assess 
their  progress  continually  as  an  integral 
part  of  learning  and  as  a guide.  Assessing 
progress  for  them  may  also  imply  active 
participation  and  responsibility  for  their 
education.  It  is  like  checking  where  they 
are  according  to  their  plan  and  where  they 
need  to  go  to  next.  An  articulated  plan  al- 
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ways  helps  as  a point  of  reference.  In  our 
program  we  make  efforts  to  help  students 
articulate  a plan  through  counseling. 

A Teacher's  Perspective.  Through 
assessment,  teachers  can  address  different 
issues  such  as:  methods  and  effectiveness 
of  their  teaching;  effectiveness  of  teaching 
styles  with  students  of  different  cultural 
and  educational  backgrounds;  the  specific 
needs  and  goals  of  a student;  their  own 
beliefs  about  what  constitutes  progress;  a 
specific  student's  abilities  and  strengths; 
and  the  student's  capabilities  and  progress 
versus  the  capabilities  and  progress  of  the 
rest  of  the  students  in  class. 

An  Intake  Worker's  Perspective.  An 
intake  worker  assessing  and  placing  stu- 
dents must  have  a pretty  good  under- 
standing of  the  range  of  classes,  as  well  as 
their  curricula.  Without  this  understand- 
ing, accurate  assessment  and  successful  • 
placement  of  students  is  extremely  diffi- 
cult. The  intake  worker  seeks  to  find  out 
whether  a particular  student  shows  signs 
of  skills  and  capabilities  usually  shown  by 
students  who  have  been  placed  in  a par- 
ticular class/level.  These  skills/capabilities 
are  considered  as  a "starting  point,"  but 
only  to  determine  placement  and  to  ensure 
a smooth  immersion  of  the  student  in  the 
system.  The  rest  is  left  in  the  hands  of  the 
counselor  and  the  teachers. 

A Counselor's  Perspective.  A counselor's 
assessment  involves  working  closely  with 
students  to  help  them  set  realistic  goals,  set 
a time  frame  in  which  these  goals  can  be 
achieved,  quantify  progress  in  and  outside 
of  class  (job  search,  resume,  referrals,  fur- 
ther education,  etc.)  and  offer  appropriate 
programmatic  response  to  expressed 
needs:  referrals  for  drop-in  babysitting, 
referral  to  another  department  for  informa- 
tion and/or  concrete  assistance,  etc.  As- 
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sessment  at  this  level  does  not  really  deal 
with  specific  language  skills,  but  rather 
how  mastering  language  skills  is  relevant 
to  the  overall  life,  educational,  and  voca- 
tional plans  of  the  students. 

An  Administrator's  Perspective.  An 
administrator's  view  about  assessment 
addresses  programmatic  concerns,  such  as 
class  size,  outcomes,  attendance,  termina- 
tions (especially  if  terminations  are  not  job 
and  education  related),  etc.  An  administra- 
tor may  use  assessment  to  (a)  measure  and 
assess  the  effectiveness  of  the  program, 

(b)  capture  and  document  outcomes, 

(c)  project  numbers  and  outcomes  for  the 
future,  and  (d)  make  the  program  (curricu- 
lum, assessment,  teaching  methods  and 
materials,  etc.)  better  respond  to  students' 
needs  for  progress.  As  an  administrator, 

I look  for  confirmation  that  the  program 
offers  its  students  what  we  believe  it  does: 
the  necessary  tools  to  achieve  their  educa- 
tional and  vocational  goals  and  to  move 
on  to  social  and  economic  self-sufficiency. 

Having  been  a teacher  not  too  long  ago, 
I still  recall  the  perspective  of  a teacher.  As 
a teacher,  I thought  of  assessment  in  a very 
limited  way:  it  had  to  do  with  my  students 
and  my  class  — not  necessarily  with  the 
whole  program  — and  it  was  as  much 
about  my  students  as  it  was  about  my 
teaching  and  the  materials  I was  using. 
Often,  I considered  assessment  a burden, 
a task  I had  to  do  and  once  completed  no- 
body would  take  another  look.  As  a result, 
I completed  the  relevant  forms  but  I was 
neither  detailed  nor  very  explanatory.  Go- 
ing back  now  to  some  of  those  assessments 
I think  they  were  poorly  written  and 
would  not  be  of  much  help  to  others. 

My  perspective  on  assessment  as  an  ad- 
ministrator is  a lot  broader  than  that  as  a 
teacher;  I have  a clearer  and  deeper  under- 
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I believe  that 
assessment  is  a 
process  that  does 
not  remain  static 
but  evolves 
together  with  the 
program  because 
it  is  the  thread 
that  connects  and 
keeps  all  the 
components  of  a 
program  in  place 
and  in  check: 
teaching, 
curriculum, 
outcomes,  goals 
and  counseling. 


standing  of  the  program  as  a whole  and 
how  its  various  components  support  and 
complement  each  other.  I also  have  a better 
picture  of  what  the  program  is  expected  to 
do  according  to  our  proposals  and  funders' 
expectations.  When  I review  a teacher's  as- 
sessment of  a student,  I expect  to  see  that 
the  teacher  knows  and  understands  a 
student's  progress.  The  student's  self-as- 
sessment and  the  teacher's  assessment 
should  be  close.  I expect  the  student  to  ex- 
press some  satisfaction  with  the  class, 
progress  made,  and  his/her  teacher.  If  the 
teacher  identifies  problem  areas,  I expect  to 
see  a plan  that  addresses  them. 

Through  the  assessment,  too,  I have  an 
indication  if  the  teacher's  work  is  effective. 
Effective  teacher's  work  means  to  me  that 
the  teacher  has  put  some  thought  into  de- 
veloping a curriculum  and  preparing  for 
his/her  class,  and  knows  the  strengths  and 
weaknesses  of  his/her  students.  Student 
attendance,  drop  out  and  termination 
rates,  and  enthusiastic  (or  not)  student 
comments  are  also  indicative  of  the  class 
in  which  active  learning  takes  place. 

Another  important  purpose  that  assess- 
ments fulfill  is  enabling  students  to  partici- 
pate in  the  educational  process  by 
assessing  their  progress.  I value  their 
assessments  and  opinions  and  take  them 
into  account  when  I assess  the  effective- 
ness and  work  of  the  program.  There  is  a 
strong  connection  between  assessment  and 
program  functions,  such  as  outcomes,  cur- 
riculum development,  and  teaching  meth- 
ods. Just  like  the  students,  the  program 
must  reflect  upon  its  operations  and  assess 
its  effectiveness  and  results.  These  results 
should  feed  bjack  into  the  program  and 
lead  to  decisions  that  will  improve  the  pro- 
gram and  benefit  its  students. 

I believe  that  assessment  is  a process 
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that  does  not  remain  static  but  evolves  to- 
gether with  the  program  because  it  is  the 
thread  that  connects  and  keeps  all  the  com- 
ponents of  a program  in  place  and  in 
check:  teaching,  curriculum,  outcomes, 
goals  and  counseling.  Unfortunately,  many 
times,  due  to  lack  of  resources  and  restric- 
tions of  time,  we  do  not  utilize  assessment 
as  a tool  for  check,  change  and  improve- 
ment to  its  fullest. 

THE  HMSC  EXPERIENCE 

Capturing  the  dynamics  of  all  assess- 
ments and  accounting  for  all  different  per- 
spectives, purposes  and  issues  may  take 
many  creative  forms,  especially  because 
most  funders  require  that  the  program  in- 
corporate initial,  on-going  and  final  assess- 
ment of  students'  accomplishments,  but  do 
not  require  any  particular  assessment  tool. 
At  the  HMSC  we  have  developed  in-house 
assessment  tools  to  capture  the  informa- 
tion we  need  and  answer  the  questions 
we  raise.  They  include  intake/ placement; 
counseling:  orientation  and  goal-setting; 
student/ teacher  conferences:  initial,  on- 
going and  final;  and  exit/ entrance  criteria 
checklist. 


Intake/Placement  Assessment 

The  Intake  worker  determines  the  level/ 
class  in  which  students  should  be  placed 
using  the  placement  test.  The  placement 
test  was  developed  by  the  program  with 
the  assistance  of  Loren  McGrail  through  a 
SABES  mini-grant  and  includes:  (a)  appli- 
cations, (b)  an  oral  interview  to  determine 
the  students'  speaking/listening  capabili- 
ties (at  this  point  initial  assessment  for 
non-literate  students  also  takes  place), 

(c)  reading  materials  to  determine  the  stu- 
dents' reading  capabilities  (reading  materi- 
als include  materials  for  different  levels 
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and  a variety  of  topics;  the  test  includes 
comprehension  questions,  writing  the 
story  in  students'  words  and/ or  respond- 
ing to  the  ideas  expressed),  (d)  writing, 
which  includes  either  responding  to  the 
reading  or  writing  about  students'  experi- 
ences, and  (e)  a grammar  test  may  be  given 
to  students  depending  on  their  level  (see 
"The  Toolkit  for  Authentic  Assessment" 
from  SABES  for  a detailed  description  and 
relevant  forms). 

This  assessment,  although  not  the  only 
one  developed,  may  be  unique  and  excep- 
tionally successful  in  placing  students:  stu- 
dents choose  to  work  with  the  readir.g(s) 
they  like  or  think  are  difficult  or  easy 
enough  for  them.  The  reading  and  writing 
materials  are  culturally  sensitive  (all  are 
pieces  of  writing  written  by  students  at  the 
HMSC)  and  so  offer  the  students  a familiar 
frame  of  reference.  The  students  also  have 
the  freedom  to  choose  the  topic  they  want 
to  write  about.  During  the  time  they  take 
the  test,  they  are  supported,  guided,  and 
helped  by  a bilingual  intake  worker.  This 
intake/placement  is  time  consuming,  tak- 
ing an  average  of  50  minutes  to  two  hours. 
The  intake  worker  needs  approximately  15 
to  30  minutes  to  read  through  and  assess 
the  level. 


Counseling 

The  second  assessment  comes  after  the 
level  of  students  is  determined  and  before 
they  enter  class.  The  Program  Counselor 
meets  with  students  individually  for 
approximately  one  hour  to  address  two  is- 
sues: program  orientation  and  goal-setting. 

Program  orientation.  This  includes  neces- 
sary information  about  other  services  that 
the  HMSC  offers,  support  services  (e.g., 
drop-in  day  care,  referral  to  other  agency 
departments  for  services),  information 
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about  elective  classes  that  the  Adult  Edu- 
cation/ESOL  Program  offers,  program 
policies,  and  specific  information  about  the 
class  the  student  will  be  placed  in  (times, 
where  the  class  meets,  and  who  the  teacher 
is). 

With  the  orientation,  students  feel  they 
are  part  of  a bigger  agency  with  additional 
available  services.  This  initial  meeting 
introduces  the  role  of  the  counselor  to  pro- 
vide support  to  the  teachers  outside  of  the 
classroom  and  provide  needed  information 
and  referrals  to  students  to  help  them 
accomplish  their  goals. 

Coal  setting.  The  counselor  completes  the 
Educational/Vocational  Plan  with  the  stu- 
dents. This  document  includes  background 
educational  and  employment  information 
about  the  students  and  includes  their  long- 
and  short-term  goals,  as  well  as  personal 
goals  (this  document  has  been  updated  to 
include  information  needed  for  the  DOE 
MIS  system). 

More  often  than  not,  students  find  it 
very  hard  to  set  goals.  It  is  especially  hard 
for  low-level  students  to  articulate  specific 
goals  besides  "to  learn  English."  The  coun- 
selor is  instrumental  in  engaging  them  in  a 
discussion  on  why  they  need  to  be  in  an 
ESOL  class  (or  in  any  other  class  for  that 
matter)  and  he  helps  them  to  articulate 
goals.  The  counselor  has  to  lead  this  dis- 
cussion carefully  (a  balance  of  articulating 
real  needs  and  imposing  them)  because  the 
students  should  set  their  own  goals,  not 
the  counselor.  The  skills  to  set  short-  and 
long-term  goals  take  a long  time  to  de- 
velop as  they  include  certain  language  that 
the  counselor  — and  the  student  — must 
master. 


The  counselor  is 
instrumental  in 
engaging 
[students]  in  a 
discussion  on  why 
they  need  to  be  in 
an  ESOL  class, 
and  he  helps  them 


to  articulate  goals. 
The  counselor  has 
to  lead  this  discus- 
sion carefully 
because  the 
students  should 
set  their  own 
goals,  not  the 
counselor. 
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Student/Teacher  Conferences 

In  these  conferences,  progress  and 
evaluation  are  discussed  by  both  parties. 
The  conferences  may  take  many  forms  de- 
pending on  the  level  of  the  students  (lower 
level  classes  may  have  more  group-like 
meetings)  and  time.  Ideally,  they  are  held 
three  times  each  session.  The  student/ 
teacher  conferences  consist  of  three  parts: 
initial,  on-going,  and  final. 

Initial.  The  first  conference  takes  place 
within  the  first  two  weeks  of  the  students' 
placement  in  a class  (either  right  after  in- 
take/ placement  or  after  movement  from 
another  class).  The  relevant  document  is 
completed  by  the  students  and  identifies 
their  specific  goals  for  the  session.  The  dif- 
ference between  this  goal-setting  and  the 
goal-setting  with  the  counselor  is  that  the 
students  think  of  goals  that  are  achievable 
in  a short  period  of  time  — the  teaching 
session.  These  goals  tend  to  be  geared 
more  around  what  they  perceive  as  weak- 
nesses and  needs. 

On-going.  The  second  conference  occurs 
around  the  middle  of  the  session,  or/ and 
any  time  that  is  deemed  necessary  or  use- 
ful for  the  teacher  or  the  students.  The 
document  has  two  parts,  one  that  is  filled 
out  by  the  students  (their  evaluation  of 
their  progress  and  goal  achievement)  and 
one  by  the  teacher  (the  student's  evalua- 
tion of  progress).  Often,  the  students  talk 
about  possible  problems  with  their  learn- 
ing, with  teaching  and  materials,  etc.  These 
problems  are  addressed  by  the  teacher  and 
possible  solutions  are  explored. 

Final.  The  last  conference  takes  place 
during  the  final  week  of  each  session.  It, 
too,  has  two  parts:  one  that  is  filled  out  by 
the  students  and  one  by  the  teacher.  The 
students  are  asked  to  evaluate  themselves 
in  terms  of  progress  and  goals  accom- 
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plished  or  not;  give  examples  of  their 
progress;  reflect  on  whether  they  have  dif- 
ferent or  additional  goals  after  attending 
the  class;  and  evaluate  the  program's  help 
and  support  in  accomplishing  these  goals. 
The  teacher  is  asked  to  provide  a formal 
evaluation  of  the  students'  progress  in 
terms  of  each  of  the  applicable  basic  skills 
(listening,  speaking,  reading,  writing,  com- 
puter literacy,  math)  and  to  provide  an 
evaluation  of  students'  attendance 
throughout  the  session. 

The  purpose  of  this  assessment  is  to  pro- 
voke discussion,  thinking,  and  evaluation 
and  become  a learning  experience  for  both 
parties.  The  teacher's  input  and  opinion  is 
as  valued  as  that  of  the  students'.  Through 
the  discussion,  the  students  become  as  re- 
sponsible for  their  learning  as  the  teacher 
is  for  his/her  teaching.  This  assessment, 
too,  is  time  consuming.  It  is  discouraging, 
however,  that  the  forms  often  do  not  cap- 
ture all  the  rich  discussion  that  they  may 
invoke  either  because  the  teachers  do  not 
write  enough/ well  or  the  students  view 
their  part  as  not  very  important  or  both.  It 
is  very  frustrating  to  read  assessments  of 
progress  that  are  poorly  written  and  that 
do  not  give  a clear  idea  of  where  the  stu- 
dents stand  in  terms  of  learning  and 
progress.  They  do,  however,  add  to  the 
incredible  amount  of  paperwork  accumu- 
lated through  the  years. 

Exit/Entrance  Criteria  Checklist 

Student /teacher  conferences  may  in- 
volve the  use  and  completion  of  the  exit/ 
entrance  criteria  checklist  when  a student 
is  ready  to  move  from  one  level  to  the  next 
one.  As  its  name  suggests,  a checklist  sum- 
marizes the  capabilities /skills  mastered  in 
a class  by  students,  and,  at  the  same  time, 
summarizes  the  capabilities/ skills  needed 
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for  those  students  to  advance  to  the  next 
class.  (There  are  as  many  checklists  as 
there  are  classes.)  The  checklist  is  com- 
pleted by  the  teacher. 

A checklist  can,  in  a short  time,  quickly 
display  the  most  important  aspects  of 
teaching  at  one  level  and  the  capabilities 
of  the  students  and,  at  the  same  time,  can 
reveal  the  capabilities  of  the  students  as  a 
starting  point  for  the  more  advanced  level. 
These  checklists  are  curriculum-driven 
as  they  include  skills  that  are  taught/ re- 
quired in  each  level.  Checklists  by  them- 
selves are  weak  evaluative  tools.  In  reality, 
a checklist  is  only  one  element  taken  into 
account  when  evaluating  students  before 


they  move  to  a more  advanced  level 
(samples  of  their  work,  especially  writing, 
and  the  initial  in-class  assessment  by  the 
new  teacher  ensure  the  right  decision). 

Despite  the  problems  mentioned  (time 
consuming,  some  resistance  on  every- 
body's part  to  keep  up  with  all  this  paper- 
work that  results  in  very  poorly  written 
assessments,  etc.),  assessment  is  a very  im- 
portant aspect  of  a program,  a crucial  tool 
for  evaluation  and  self-evaluation,  and  an 
important  part  of  teaching  because  it 
touches  all  aspects  of  a program  and  can 
be  used  to  confirm  and  validate  findings 
about  the  pf ogr am,  its  services,  and  its 
students. 


BT 


[Author  's  note:  "I  wanted  to  acknowledge  Alison  Simmons  for  her  assistance  in  the  thinking  and 
writing  of  this  article. "] 
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STUDENT  EDUCATIONAL  VOCATIONAL  PLAN 

Student's  Name: 

Counselor's  Name: 

Date  of  Orientation/ Counseling: 


I.  PLACEMENT  IN  PROGRAM 

Native  Language  Literacy: 

ESOL  Class: 

Pre-EDP/EDP: 

Counseling/Orientation  (date): 

Math: 

Computer  Training: 

Other  (specify): 

II.  BACKGROUND 

1 . Education: 

2.  Previous  Job  Experience: 

3.  Skills: 

III.  POSSIBLE  REFERRAL  (from,  to,  date,  reason) 

IV.  STUDENT  GOALS 

1.  Educational  Goa/s 

a.  Complete  a class 

b.  Speaking 

Communicate  more  effectively  in  English 

-understand  and  feel  comfortable  in  a conversation 
-have  a conversation  with  other  people 
-communicate  with  people  on  the  phone 
-understand  the  TV 
-understand  movies 
-understand  the  news 

732 


ERiC 


'Mmim.mmM  mJRES  IN  ASSESSMENT 


Volume  TO:  December  1997 


Volume 
10  • 21 


c.  Reading 


Be  able  to: 


-read  a newspaper 

-read  a book 

-read  maps,  calendars 

-look  up  a word  in  a dictionary 

-find  information  in  the  phone  book 


-call  91 1 effectively 
-make  doctor's  appointments 


-give  personal  information  (name,  address,  phone  #) 
-letters  (to  friends,  for  business) 


f.  Obtain  EDP 

-Complete  tasks  (specify) 

g.  Obtain  GED 

-Pass  some  tests  (specify) 

h.  Complete  some  adult  HS  credits 

i.  Enroll  in  Post-Secondary  Education 


a.  Enter  vocational  training  (specify) 

b.  Gain  Employment 

c.  Obtain  Job  Advancement 

d.  Perform  New  Job  Requirements 


d.  Survival 


Be  able  to: 


e.  Writing 


Be  able  to: 


2.  Vocational  Goals 
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e.  Be  Removed  from  Public  Assistance 

f.  Fill  out  applications 

3.  Personal  Goa/s 

a.  Read  more  to  children 

b.  Increase  parent/child  interaction 

c.  Help  children  with  homework 

d.  Enroll  in  activities  that  support  public  school 

e.  Receive  preventive  health  care 

4.  Community 

a.  Enroll  in  Civics  class 

b.  Receive  US  Citizenship 

c.  Register  to  vote  for  first  time 

d.  Involved  in  activities  within  own  community 

e.  Incarcerated  adult  return  to  society 

V.  NEEDS  DETERMINED  TO  ACHIEVE  GOALS 

1. 

2. 

3. 

4. 

5. 
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VI.  STUDENT  OUTCOMES  (to  be  completed  upon  termination) 

1 . Termination  Date: 

2.  Level  started  (include  SPL): 

3.  Made  progress  within  same  level  (include  SPL): 

4.  Highest  level  completed  (include  SPL): 

5.  Found  Employment 

(Place,  Job  Title,  Phone  Number,  Starting  Date,  Salary) 


(Previous  Job  Title,  Current  Job  Title,  Finished  Relevant  Training  -where?-) 

7.  Entered  Skills  Training 
(Where,  for  how  long) 

8.  Entered  Higher  Education 
(Where,  what  program) 

9.  Became  Citizen 


(When) 


VII.  ADDITIONAL  COMMENTS 
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STUDENT-TEACHER  CONFERENCE 

Student's  Name: ; Date:. 

Teacher's  Name: ■ 

Class  Level: Time:. 

INITIAL  EVALUATION 

1 .  Why  are  you  in  this  class?  Why  do  you  want  to  learn  English? 


2.  How  many  years  of  education  do  you  have  altogether? 


3.  Do  you  have  a high-school  diploma  (from  Haiti  or  another  country)? 


4.  Do  you  have  any  skills  (for  example,  do  you  know  howto  use  a typewriter  or  a computer, 
etc.)? 


5.  If  you  want  to  go  to  college,  have  you  decided  what  you  want  to  study? 


6.  Do  you  have  any  questions  about  this  program  or  your  class? 
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STUDENT-TEACHER  CONFERENCE 

Student's  Name: Date: 

Teacher's  Name: 

Class  Level: Time: 

ON-GOING 

1 . Do  you  feel  that  you  are  making  progress?  Do  you  feel  you  are  making  progress  in: 

-Listening  and  understanding 

-Speaking 

-Writing 

-Reading  and  understanding  (readings  inside  and  outside  of  class) 

-Pronunciation 

-Computers 

-Math 

-Grammar 

-Other 

2.  If  you  feel  you  are  making  progress,  how  do  you  understand  it?  (Give  some  examples, 
please). 

3.  If  you  feel  you  are  not  making  progress,  why  is  it  so? 

4.  Is  there  anything  you  want  to  change  in  your  class? 

5.  Is  there  anything  you  want  to  do  more  in  class? 

6.  Do  you  have  any  questions  about  this  program  or  your  class? 
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STUDENT-TEACHER  CONFERENCE 

Student's  Name: 

Date: 

Teacher's  Name: 

Class  Level: 

Time: 

FINAL  EVALUATION 

Proaress  Assessment 

For  Students:  In  what  areas  did  you  make  progress?  How  do  you  know?  (Give  examples 
from  your  class  and  outside  of  class,  please). 

For  Teachers:  In  what  areas  did  the  student  make  progress? 

How  do  you  know? 

Reading: 

Writing: 

Speaking: 

Listening: 
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Pronunciation: 


Grammar: 


Word-Processing: 


Math: 


Other: 


Attendance:  Regular  (75%)  Yes  No  (If  no,  please  give  reason(s)} 
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Overcoming  Cultural  Barriers 
of  a Job  Interview 


by 

Judy  Chau 


Asian  American 
Civic  Association 
Boston,  MA 


Never  overlook  the  obvious.  This 
is  what  I continue  reminding 
myself.  After  three  years  of 
coaching  students  in  a prevocational  edu- 
cation program  to  prepare  for  job  inter- 
views, particularly  "white-collar"  job 
interviews,  I find  myself  backtracking  and 
focusing  much  more  with  students  on  the 
basic  presentation  aspects  of  interviewing. 

While  our  program  is  open  to  the  non- 
native speaking  population  of  Greater  Bos- 
ton, the  typical  profile  of  an  Asian  Ameri- 
can Civic  Association  (AACA)  Prevoca- 
tional Skills  student  is  a Chinese  or  Viet- 
namese female,  in  her  late  20s  to  early  30s, 
who  has  worked  as  a stitcher,  cashier  or 
waitress  in  a Chinatown  business  since 
arriving  in  the  United  States.  The  students 
are  high-beginners  or  low-intermediate 
level.  The  average  current  length  of  resi- 
dency in  the  U.S.  is  two  or  three  years. 

Most  students  have  a very  solid  work 
history  in  their  country,  having  worked  for 
a single  employer  for  up  to  ten  years.  Many 
have  had  excellent  skills,  as  teachers,  book- 
keepers, secretaries,  bank  tellers,  business 
owners,  doctors,  tour  guides,  or  hotel 
clerks.  They  are  mature,  with  a stable  work 
history  and  many  transferable  skills.  The 
main  barriers  are  limited  English,  difficult 
acculturation,  and  a lack  of  computer 
skills,  confidence,  and  - for  some  - adequate 
daycare.  Most  are  very  employable  indi- 
viduals who  typically  do  not  see  them- 
selves as  employable  beyond  Chinatown. 
This  is  one  great  barrier  - especially  since 
many  usually  summarize  their  goal  as  "I 
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want  a good  job,  an  office  job". 

Enrollment  typically  consists  of  twenty 
students  who  are  divided  into  two  equal 
groups.  Classes  run  16  hours  per  week. 
Although  not  an  office  skills  training  pro- 
gram, the  students  are  taught  the  basics  of 
keyboarding  and  basic  word  processing 
functions  of  Microsoft  Word.  They  practice 
math  and  learn  the  English  necessary  for 
the  mathematical  skills  which  they  possess. 
They  study  grammar  for  two  hours  per 
week,  but  ESL  is  intertwined  in  all  course 
matter.  In  addition  to  aiding  students  with 
these  basic  skills,  the  task  of  the  three 
prevocational  teachers  is  to  assist  them  to 
accept  themselves  as  employable  outside 
of  Chinatown.  They  participate  in  self- 
assessment  activities  to  recognize  their 
strengths.  The  Case  Manager  works  with 
the  students  on  an  ongoing  basis  for  five 
months,  both  in  workshops  and  individu- 
ally, to  develop  goals  and  follow  their  ac- 
tion plan. 

My  responsibility  is  divided  into  ten 
weeks  of  conversation,  culture  topics,  writ- 
ing skills,  and  pronunciation,  along  with 
ten  weeks  of  introducing  U.S.  culture  as  it 
relates  to  the  world  of  work.  The  greatest 
challenge:  to  prepare  students  for  a "white- 
collar"  job  interview.  The  goal  of  the  en- 
rollees  is  to  either  find  a full-time  job  after 
the  five-month  class  period,  or  to  enroll  in 
a full-time  training  program.  Typically,  up 
to  80%  opt  for  the  training  program;  this 
may  be  the  Office  Systems  Training  Pro- 
gram at  AACA,  or  others  throughout  the 
city  of  Boston. 
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Herein  lies  the  challenge.  Within  those 
months,  the  students  must  feel  confident 
and  ready  for  either  a job  or  training. 
Whether  preparing  for  a training  program 
interview,  or  preparing  for  a job  interview 
- white-collar  or  not  - the  challenges  remain 
the  same:  the  candidate  must  appear  confi- 
dent, describe  and  "sell"  their  skills  and 
experience,  and  explain  their  motivation 
and  goals. 

The  problem  lies  in  the  fact  that  most 
do  not  feel  confident,  cannot  adequately 
describe  their  skills,  have  no  knowledge  of 
the  Boston  job  market  and  have  not  devel- 
oped long-term  career  plans  which  they 
could  express  to  a Human  Resources  pro- 
fessional. So,  on  the  larger  scale,  profi- 
ciency in  English,  a realistic  view  of  the 
current  office  job  market  in  Boston,  the 
necessary  keyboard  and  computer  skills 
to  obtain  an  office  job,  and  a plan  are  para- 
mount to  achieving  their  goals. 

Less  evident,  yet  equally  as  important 
to  the  process,  are  the  presentation  skills 
which  native  speakers  of  English  are 
coached  on  at  University  Career  Centers, 
Massachusetts  One-stop  Career  Centers, 
or  by  employment  specialists.  They  are  the 
handshake,  volume,  eye  contact,  and  the 
all  important  ability  to  "sell  yourself"  at 
every  opportunity.  These  are  the  deeper, 
cultural  aspects  of  Western-style  inter- 
viewing which  my  students  repeatedly 
inform  me  go  against  cultural  propriety. 

In  addition,  in  the  case  of  students  from 
mainland  China,  jobs  were  assigned  by  the 
government,  and  they  have  never  experi- 
enced what  we  know  as  a job  interview. 
The  lack  of  experience  with  interviewing, 
and  the  cultural  differences  are  very  evi- 
dent as  they  come  to  interview  for  a slot 
in  the  Prevocational  Program. 

As  a Prevocational  Instructor,  I am  a 
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member  of  the  seven-person  Employment 
and  Training  team  at  AACA.  The  entire 
team  participates  in  the  intake  process. 
Applicants  may  be  directed  in  one  of  four 
ways  in  AACA.  They  may  be  accepted  to 
either  Prevocational  Skills  or  to  Office  Sys- 
tems. Candidates  who  need  much  more 
remedial  English  are  referred  to  ESL  class. 
Those  who  test  with  high  conversational 
skills  and  extensive  office  experience  in 
their  homeland  are  referred  to  the  AACA 
Job  Developer. 

During  the  intake  process  of  potential 
Prevocational  or  Office  System  students, 
wq  ole  assessing  xuaTiy  factors,  rrom  eiigi- 
bility  requirements  including  low-income 
status,  residency,  and  having  a work  per- 
mit, to  their  English  level,  and  to  their 
commitment  to  the  objectives  of  the  pro- 
gram. With  respect  to  selecting  who  stud- 
ies in  which  program,  it  often  comes  down 
to  our  determination  of  their  apparent  un- 
derstanding of  cultural  expectations  in  the 
intake  interview  and  their  confidence  in 
themselves.  Candidates  who  come  across 
professionally,  who  smile,  give  a strong 
handshake,  have  good  eye  contact,  and  are 
willing  to  try  to  define  their  work  history 
will  typically  be  accepted  to  office  skills. 
Realistically,  by  learning  the  appropriate 
office  skills,  they  will  be  job-ready  in  five 
months.  In  general,  these  candidates  have 
lived  for  some  time  in  the  United  States, 
and  have  somehow  become  acculturated  to 
many  of  these  interviewing  factors.  They 
still  have  to  strengthen  these  skills,  but 
they  are  well  on  their  way. 

The  typical  Prevocational  candidate  will 
rarely  extend  their  hand  to  the  teacher/ in- 
terviewer. We  extend  our  hand  first.  The 
handshake  which  we  receive  is  the  classic 
"dead-fish"  handshake,  described  in  how- 
to interview  books.  It  is  limp,  with  little  or 
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no  grasp.  Often  it  is  a two-fingered  shake, 
or  "a  slider,"  which  slips  through  our  fin- 
gers before  we  have  a chance  to  grasp  it. 
During  the  first  week  of  class,  we  begin  to 
practice  the  handshake. 

It  is  approached  from  the  cultural  as- 
pect first.  Students  learn  that  it  is  appropri- 
ate protocol  within  the  business  world  to 
do  so.  Women  are  assured  that  they  are 
crossing  no  boundaries  to  shake  hands, 
especially  with  a man.  They  learn  the  cul- 
tural interpretation  of  a weak  handshake 
that  human  resource  professionals  empha- 
size: a weak  handshake  is  a sign  of  either  a 
weak  character,  or  a weak  body  — or  both. 
Either  way,  these  are  less  than  desirable 
traits  in  a employer's  confident,  compe- 
tent, healthy  workforce. 

Three  years  ago,  I felt  that  a bit  of  prac- 
tice and  a cultural  explanation  would  be 
enough  to  make  students  aware  of  the 
need  for  a great  grasp.  I felt  as  though  I 
would  belittle  my  students  to  continue  to 
review  this  point.  I have  since  discovered 
how  culturally  difficult  this  seemingly 
simple  act  can  be.  The  students  have  lis- 
tened to  many  human  resource  specialists 
who  have  come  as  guest  speakers  and 
have  emphasized  this  point.  They  have 
practiced  with  the  H.R.  specialists,  they 
practice  at  least  once  a week  in  class,  and 
during  the  19th  week  of  the  program,  as 
students  actually  experience  their  first 
complete  mock  interview  with  a true  inter- 
viewer rather  than  a teacher,  most  will 
have  the  handshake  down.  There  will  still 
be  a few  limp  hands.  My  mistake  three 
years  ago  was  believing  that  I would  insult 
my  students  by  emphasizing  this  point.  I 
discovered  that  I was  not  doing  them  a 
favor  by  failing  to  demand  consistently 
strong  handshakes. 

Clearly,  a firm  handshake  is  only  a piece 
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of  the  outside  package.  Equally  important 
is  for  the  eyes  to  meet.  Maintaining  eye 
contact  throughout  a thirty-minute  inter- 
view takes  practice  for  any  person  who 
may  be  less  than  comfortable  in  an  inter- 
view. Coupling  that  with  the  fact  that  in 
Chinese  and  Vietnamese  culture  direct  eye 
contact  in  a formal  situation  such  as  a class 
or  a job  interview  is  seen  as  disrespectful, 
this  is  our  second  obstacle  to  overcome.  In 
the  initial  intake  interview  for  the  Prevoca- 
tional  Program,  eye  contact  is  no  more  than 
the  fifty  percent  range. 

And  once  again,  it  is  first  approached 
through  a cultural  perspective.  Students 
list  the  implications  of  giving  direct  eye 
contact  in  a formal  situation.  They  inform 
me  that  to  do  so  is  to  be  an  affront.  Words 
that  they  include  are  "rudeness"  "arro- 
gance" or  "a  challenge".  They  are  then 
introduced  to  the  American  take  on  lack 
of  eye  contact  — basic  shyness  and  insecu- 
rity, possible  boredom  or  lack  of  interest, 
perhaps  avoidance  of  the  truth  to  a par- 
ticular question.  The  students  are  very  sur- 
prised by  these  last  two  negative  interpre- 
tations, but  no  miracles  of  eye  contact 
come  strictly  with  knowledge. 

Throughout  the  twenty  weeks,  the  stu- 
dents practice  pair  activities  in  which  they 
observe  each  other's  eye  contact.  Not  being 
a formal  instance  such  as  speaking  to  a 
teacher  or  an  interviewer,  this  is  much 
easier.  Although  each  student  spends  a 
total  of  eight  hours  per  week  with  me,  for 
twenty  weeks,  they  usually  still  consider 
contact  with  me  as  "formal".  On  an  ongo- 
ing basis  I may  sit  down  with  a particular 
student  in  class  one-on-one,  and  have  a 
chat.  During  the  chat,  I will  follow  their 
eyes  — up  to  the  ceiling,  down  to  the  floor, 
toward  that  invisible  spider  on  the  wall. 
This  is  done  in  a good-humored  way,  and 
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the  students  have  some  idea  of  how  much 
their  eyes  travel  in  a one-minute  conversation 
At  the  culmination  of  the  job  search 
skills  segment  of  the  Prevocational  Pro- 
gram, the  students  participate  in  a 15  to  20 
minute  videotaped  mock  interview,  gener- 
ally with  one  of  several  Human  Resources 
professionals  who  cooperate  with  AACA. 
Until  the  eighteenth  week,  when  the  stu- 
dents have  an  opportunity  to  view  them- 
selves on  the  video,  eye  contact  remains  a 
challenge.  I find  that  it  improves  after  the 
students  take  their  taped  interview  home 
to  review.  Seeing  is  believing. 

Volume  is  another  "sales  point"  which 
we  work  on.  Again,  sitting  face  to  face,  in 
a formal  situation,  speaking  with  volume 
and  emphasizing  points  emphatically, 
might  indicate  a certain  boisterousness,  ar- 
rogance or  disrespect.  Again,  this  point  is 
worked  on  throughout  the  twenty  weeks. 

The  students  bring  a blank  cassette  to 
class  from  week  15  onward.  We  spend 
the  good  part  of  three  weeks  practicing 
approximately  35  common  interview  ques- 
tions. I act  as  interviewer,  the  classmates 
observe  and  make  notes  on  a critique  form. 
When  initially  playing  back  the  tape,  stu- 
dents note  they  hear  me  very  clearly,  but 
have  trouble  hearing  themselves.  We  are 
sitting  equidistant  from  the  recorder.  This 
method  seems  to  help  significantly,  and 
students  self-improve  day  by  day;  most  are 
able  to  match  the  volume  and  energy  of 
the  interviewer.  Others  improve  after  they 
hear  the  same  problem  with  volume  re- 
flected in  the  videotape.  We  do  our  best  to 
improve  awareness,  and  to  increase  confi- 
dence, but  someone  who  is  inherently  shy 
may  continue  to  be. 

Although  the  handshake,  eye  contact, 
and  volume  are  somewhat  determined  by 
culture,  the  main  cultural  barrier  is  the 
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central  requirement  of  Western-style  inter- 
viewing: "selling  yourself."  In  cultures 
where  humility  is  a virtue,  the  concept 
of  recounting  past  accomplishments  and 
emphasizing  strengths  is  a huge  obstacle. 
The  concept  is  a challenge.  The  English 
vocabulary  is  a challenge.  Believing  what 
you  say  is  a challenge. 

Students  must  be  convinced  that  their 
work  history  in  their  home  country  is  im- 
portant. We  find  during  the  initial  intake 
interview  that  candidates  tell  us  only 
about  their  work  experience  in  the  United 
States,  as  if  what  went  before  is  not  valid. 
This  is  especially  true  for  former  profes- 
sionals who  realize  that  they  may  not  prac- 
tice their  previous  profession  again,  at 
least  not  on  the  same  level. 

To  first  introduce  them  to  which  skills 
are  valued  and  sought  in  the  world  of 
office  work,  the  students  spend  a week 
learning  the  want  ad  abbreviations,  and  re- 
viewing the  Boston  Globe  office  job  ads.  The 
purpose  of  the  exercise  is  twofold:  first,  to 
view  the  current  job  market  realistically; 
second,  to  see  which  "hard  and  soft"  skills 
are  being  sought. 

They  can  clearly  see  in  newsprint  which 
hard  skills  or  computer  skills  are  being 
sought.  This  helps  them  set  some  concrete 
training  goals.  In  addition,  once  they  learn 
that  basic  soft  skills  such  as  teamwork,  ini- 
tiative, hard  work,  cooperation,  honesty, 
enthusiasm  are  listed  time  after  time  in 
expensive  advertising  space,  they  begin 
to  believe  that  they  have  something  to  sell. 

We  work  for  one  week  to  develop  the 
vocabulary  of  such  "soft  skills".  Students 
practice  giving  vivid  examples  both  in 
writing  and  orally.  For  example,  one 
former  student  wrote,  "I  like  to  take  initia- 
tive. I know  that  in  the  past,  taking  initia- 
tive helped  me  to  advance  in  my  job.  One 
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day,  when  I was  working  at  an  electronics 
company,  the  assembly  line  stopped  work- 
ing. My  coworkers  sat  on  the  floor  playing 
cards,  waiting  for  the  line  to  start.  I 
thought  this  was  a good  opportunity  to 
learn  something.  I asked  my  supervisor  if 
I could  see  the  other  department  working. 

I watched  the  other  worker  for  more  than 
an  hour,  and  I thought  I could  do  that.  The 
next  day,  I told  my  supervisor  that  I was 
sure  I could  do  that.  The  next  week,  I got 
a promotion  to  that  department.  I don't 
think  that  you  can  advance  without  doing 
extra  things." 

The  next  week  is  spent  on  "functional 
or  transferable  skills,"  or  the  "I  am  good 
at. skills.  Functional  skills  are  things  one 
can  do  well.  We  do  the  same  review  of  the 
want  ads,  and  look  for  key  words  such  as 
"organize,  analyze,  instruct,  sell,  convince, 
motivate,  prioritize,  calculate,  handle  mul- 
tiple tasks,  plan,  mediate  problems." 

Again,  we  spend  a week  for  students  to 
develop  the  vocabulary  appropriate  to 
their  experience,  develop  personal  ex- 
amples, and  practice  pronunciation. 

The  next  three  weeks  consist  of  more 
vocabulary  development  and  affirmation 
of  their  previous  skills  as  something  valid. 
This  is  done  with  the  help  of  the  Job  Devel- 
oper and  the  Case  Manager.  The  three  of 
us  spend  significant  individual  time  to  de- 
velop vocabulary  for  the  third  component, 
the  "job-specific"  skills.  We  trace  their 
work  history  with  a chronological  form 
and  choose  the  most  appropriate  terminol- 
ogy to  accurately  describe  their  experience. 
Having  no  experience  as  a seamstress,  an 
accountant,  or  an  acupuncturist,  I cannot 
pull  terminology  from  the  top  of  my  head. 
Developing  occupational  or  professional 
vocabulary  is  a team  effort,  using  occupa- 
tional resource  books. 
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To  further  reinforce  and  practice  this 
vocabulary,  the  students  will  fill  out  up  to 
ten  Boston-area  job  applications  in  class. 
They  will  then  develop  a resume,  and  will 
format  it  in  computer  class.  They  will  write 
basic  cover  letters  in  response  to  mock  ad- 
vertisements. Through  all  of  these  media, 
the  students  become  comfortable  with  the 
vocabulary,  and  hopefully  they  also  begin 
to  see  that  they  do  have  valid,  valued 
skills. 

Weeks  ten  to  fifteen  have  carried  us 
through  the  job  market  exploration,  skills 
self-assessment,  job  applications,  resumes, 
and  cover  letters.  By  this  time  they  should 
have  a great  handshake,  make  consistent 
eye  contact,  project  their  voices  well,  be- 
lieve that  they  have  skills  to  offer,  know 
how  to  describe  their  skills,  and  are  realis- 
tic about  which  "hard"  skills  they  must 
learn  in  a future  training  program. 

Weeks  sixteen  to  eighteen  consist  of 
daily  practice  of  30  to  40  common  inter- 
view questions,  including  the  areas  of 
small  talk,  education  background,  work 
experience,  work  style/ personal  traits, 
career  plans,  and  hiring  details,  such  as 
schedules.  They  have  a nightly  homework 
assignment  to  prepare  complete  answers  to 
three  or  four  interview  questions.  The  fol- 
lowing day,  as  one  student  is  interviewed 
on  their  personal  cassette  tape,  the  others 
observe  and  critique  each  other. 

By  this  time,  they  have  become  very 
aware  of  the  completeness  of  answers,  the 
effectiveness  of  personal  examples,  vol- 
ume, eye  contact,  posture,  and  general 
"sales  ability."  Having  been  together  for 
almost  20  weeks,  they  are  comfortable  with 
each  other  and  generally  very  supportive, 
yet  they  are  often  more  critical  of  each  oth- 
ers performance  than  I might  be.  This  just 
indicates  their  awareness,  and  I love  when 
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I see  that. 

They  are  ready  for  a full  trial  run.  Mem- 
bers of  the  AACA  Employment  and  Train- 
ing Advisory  Board  are  called  in.  These  are 
most  often  Human  Resource  specialists 
from  major  area  employers  such  as  Fleet 
Services,  BankBoston,  New  England  Medi- 
cal Center,  MetLife.  They  interview  those 
whose  English  skills  are  in  the  higher 
range,  and  who  have  a great  deal  of  com- 
fort by  now  with  the  interview  process. 
Those  who  are  still  struggling  with  English 
and/or  confidence  are  interviewed  by 
Board  members  with  Adult  Education  and 
ESL  affiliations.  We  do  not  tel!  this  to  the 
students,  so  not  to  single  them  out.  Clearly, 
these  interviewers  have  strategies  to  ease 
them  through  the  first  interview. 

By  now,  the  students  have  been  observ- 
ing each  other  for  at  least  three  weeks.  The 
mock  interview  takes  place  in  a classroom 
observed  by  the  teacher  and  up  to  10  class- 
mates, and  it  is  videotaped.  It  is  a highly 
artificial  situation,  but  we  hope  the  stu- 
dents will  benefit  from  multiple  forms  of 
feedback.  The  students  seem  to  consider  it 
a rite  of  passage,  and  say  that  it  is  benefi- 
cial. Because  there  is  no  real  job  at  stake, 
they  just  consider  it  a good  opportunity  to 
practice. 

During  the  interview,  classmates  fill  out 
peer-critique  forms  which  are  in  the  form 
of  a checklist.  In  fact,  they  have  been  doing 
this  informally  during  the  past  three 
weeks.  They  pass  these  to  the  interviewee 
after  the  class.  At  the  culmination  of  the  in- 
terview, the  interviewee  gives  a self-cri- 
tique, describing  the  basic  level  of  comfort, 
explaining  which  questions  were  difficult, 
and  suggesting  how  they  might  improve 
those  answers  in  the  future.  By  the  stu- 
dents' own  critique,  it  is  clear  how  aware 
they  are  of  themselves.  When  they  can 


laugh  and  suggest  improvements  for 
themselves,  I know  that  they  are  on  the 
way  to  successful  interviews  in  the  future. 
This  is  a great  sign  of  success. 

The  interviewer  then  gives  a critique, 
explaining  the  strengths  and  weaknesses 
of  the  interview,  and  makes  suggestions. 
The  final  critique  is  done  by  the  teacher 
who  focuses  more  on  pronunciation  diffi- 
culties with  pivotal  words,  or  grammar 
which  may  have  impeded  understanding.  I 
write  those  specific  words  down,  and  they 
may  ask  me  to  practice  with  them,  and  to 
put  it  on  their  audio  cassette  later. 

At  twenty  minutes  per  interview,  pius 
time  to  critique,  interviews  may  take  three 
or  four  days  to  complete.  The  students  ap- 
plaud each  other  and  release  a collective 
sigh  of  relief.  I then  copy  the  videotape 
onto  VHS,  and  students  circulate  that 
copy,  often  making  their  own  tape. 

Cycle  after  cycle,  students  tell  me  how 
they  review  their  audiocassettes  and  video 
cassette,  especially  as  they  begin  to  gradu- 
ate from  other  training  programs  and  pre- 
pare for  interviews.  They  have  also  used 
the  videotapes  to  help  instruct  friends  and 
family  members  about  interviewing. 

By  spending  almost  10  weeks  on  the  ac- 
tual process  of  skills  self-assessment,  job 
search  skills,  and  interviewing  we  hope  to 
prepare  students  for  two  options.  The  first 
is  to  seek  employment  directly  from  the 
Prevocational  Program.  Of  course,  unless 
they  have  work  experience  in  a related 
field,  and  a certain  competency  with  En- 
glish, they  will  be  seeking  more  basic  en- 
try-level service  jobs.  Secondly,  for  those 
who  plan  to  continue  on  with  training  pro- 
grams, we  hope  to  give  a comprehensive 
ESL-oriented  view  of  the  employment 
search  process  which  will  be  used  in  con- 
junction with  the  more  rapid-pace  presen- 
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tation  of  such  skills  in  other  city-wide  pro- 
grams where  they  may  be  studying  with 
native-English  speakers. 

Clearly,  a non-native  speaker  faces  the 
same  interview  challenges  as  a native 
speaker.  They  face  competition,  nervous- 
ness, insecurity,  and  the  need  to  prepare. 


Coupled  with  linguistic  and  cultural  barri- 
ers, the  prospect  can  be  overwhelming.  In 
preparing  students  for  the  process,  we  as 
ESL  teachers  should  not  be  uncomfortable 
to  emphasize  the  basics.  If  we  do  not,  then 
who  will? 
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Peer  Evaluation  Form  for 


introduces  self  & handshake 

posture 

energy 

eve  contact 

volume 

makes  small  talk 

talks  about  education 

talks  about  past  employment 

explains  interest  in  position 

talks  about  strengths  and  work  style 

explains  long/short-term  goals 

uses  good  examples  to  clarify  points 

asks  appropriate  questions  about  job 

says  "thank  you"  & shakes  hands 

in  general,  sells  herself/himself  well 


Comments 
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Where's  the  EGAP  These  Days? 


Maybe  you  remember  when 

Martha  Germanowski  designed 
something  called  an  Educa- 
tional Goals  Assessment  Packet  for  learn- 
ers in  adult  education  classes  for  the 
homeless.  The  EGAP  was  designed  to  get  a 
big  picture  of  students'  interests,  to  help 
learners  focus  on  their  goals,  and  to  show 
their  progress.  It  had  an  extensive  checklist 
of  reading,  math  and  life  skills,  a goals 
page,  a monthly  review  page,  and  a daily 
log. 

This  "big  picture"  benefited  the  teacher 
in  her  planning  for  a multi-level  class.  It 
benefited  learners  who  could  choose  their 
goals,  see  progress  on  those  specific  goals, 
and  then  get  positive  feedback  on  their 
daily  work.  Like  all  teacher  tools,  parts  of  it 
wore  out,  parts  didn't  work  well,  and  other 
parts  proved  to  be  "keepers". 

A few  years  after  its  appearance,  the 
EGAP,  like  Martha  G.,  is  in  transition. 

Now,  as  Martha  Jean,  I am  considering 
more  EGAP  modifications. 


WHAT  HAS  HAPPENED 
OVER  THOSE  FIVE  YEARS? 


by 

Martha  Jean 


Community  Action  Inc. 
Haverhill,  MA 


o 


I started  using  the  EGAP  in  my  Pre-GED 
and  GED  classes  because  it  gave  those 
learners  a sense  of  their  own  accomplish- 
ments, possible  goals,  and  the  same  posi- 
tive feedback.  I added  and  then  removed  a 
Reading  Interest  Checklist  that  didn't  tell 
me  more  than  an  existing  intake  question 
about  how  well  a student  reads.  I kept  and 
modified  the  Daily  Log,  Monthly  Goals 
Review,  and  the  Educational  Goals  Plan 
pages. 

Because  of  my  continual  urge  to  make 
the  EGAP  more  visually  pleasing,  over  the 
years  I've  made  some  simple  changes  on 
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all  the  pages,  aided  by  computers.  The 
lines  have  been  removed  and  a clearer 
font  used.  Bold  and  italicized  headers 
have  been  added.  Because  the  options  of 
"I  know/ do  this,"  "I  would  like  to  know 
more  now/later,"  "I  understand  this  and  I 
am  ready  for  the  next  steps,"  added  to  stu- 
dent confusion,  the  latest  model  reads  sim- 
ply: I KNOW  THIS and  I WANT  TO 

KNOW  MORE  ABOUT  THIS_.  Students 
are  told  to  check  only  what  is  important  to 
them.  Progress  is  reported  in  the  Daily 
Log. 

This  reads  as  if  I am  using  the  EGAP.  I 
am  pleased  to  say  that  many  teachers  have 
used  it  or  modified  it  for  different  class- 
room use  over  the  years.  But,  I didn't  use 
it  at  £ill  this  past  year. 

WHY  NOT? 

This  year  our  students  had  extensive 
intakes  and  testing  for  the  DOE  SMARTT 
System.  It  seemed  excessive  to  burden 
them  with  so  much  paperwork  before  they 
even  got  to  be  involved  in  learning.  So,  I 
set  the  EGAP  aside  to  think  about  how  to 
make  it  most  useful  in  my  GED  and  multi- 
level homeless  education  classes. 

I also  became  part  of  a teacher  research 
group  about  multiple  intelligence.  That 
added  a whole  new  perspective  about 
how  I might  want  to  help  students  know 
themselves,  their  strengths,  and  what  they 
already  know.  I had  a lot  to  consider  for 
my  next  EGAP  remake. 

WHAT  IS  MY  THINKING  ABOUT 
THE  "FUTURE  EGAP"? 

I don't  want  to  overwhelm  new  class 
members,  so  I am  going  to  break  the  EGAP 
into  smaller  parts  to  be  completed  by  stu- 
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dents  over  some  weeks.  I don't  plan  on  re- 
moving anything;  students  and  I like  what 
it  indudes.  But,  to  the  checklist  I will  add 
skills  related  to  music,  movement,  nature, 
spatial  understanding,  interpersonal  and 
intrapersonal  ability.  My  multiple  intelli- 
gences research  has  shown  me  that  these 
are  equally  valuable  areas  of  knowledge. 
My  research  experience  has  also  reminded 
me  how  learners  benefit  from  time  to  talk 
about,  share,  and  consider  their  options. 
The  EGAP  can  be  more  than  a checklist  if 
time  is  given  to  explore  the  choices  learn- 
ers make.  This  will  support  learners'  work 
to  reach  their  goals. 


The  EGAP  was  designed  as  and  at  its 
core  has  remained,  a tool  for  teachers  and 
learners  to  identify  what  learners  know 
and  what  they  dream  of  knowing.  There  is 
a do-able  goals  plan  and  a daily  chance  to 
communicate  successes,  failures,  hopes, 
and  fears.  There  is  a place  for  goals  review 
and  revision.  Those  have  remained  the 
same. 

Changes  happen  whenever  I or  another 
teacher  asks,  "How  is  this  working  for  the 
learners?"  "Is  this  leading  to  some  positive 
learning  or  is  this  more  burdensome  pa- 
perwork?" or  "How  can  I make  this  work 
better?"  Those  questions  continue  to  make 
the  EGAP  what  it  is  and  what  it  can  become. 

0 


My  research 
experience  has 
also  reminded  me 
how  learners 
benefit  from  time 
to  talk  about, 
share,  and 
consider  their 
options. 


ERIC 

^“aapjliime  10:  December  1997 


ADVENTURES  IN  ASSESSMENT 


Volume 
10  • 38 


EDUCATIONAL  GOALS  PLAN 

THESE  ARE  THE  EDUCATIONAL  GOALS  I WILL  WORK  ON  WHILE  I AM  IN  THE 
COMMON  GROUND  CLASSES: 

1.  


2. 


3. 


4. 


5. 


I HAVE  READ,  OR  SOMEONE  HAS  READ  TO  ME,  MY  EDUCATIONAL  OPTIONS. 
I HAVE  CHOSEN  THE  GOALS  I WOULD  LIKE  TO  START  WORKING  ON  IN  EACH 
CLASS. 

SIGNED  DATE 
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EDUCATIONAL  GOALS  ASSESSMENT  PACKET 


PUT  A 


NEXT  TO  YOUR  CHOICES 


I WANT 

I KNOW  TO  KNOW  I KNOW 

THIS  MORE  ABOUT  THIS 

THIS 


I WANT 
TO  KNOW 
MORE  ABOUT 
THIS 


COMPREHENSION  SKILLS 

□ READ  AND  WRITE  MY  NAME  □ 

□ READ  AND  WRITE  MY  ADDRESS  □ 

□ READ  AND  WRITE  MY  PHONE  NUMBER  □ 

□ READ  AND  WRITE  MY  SOCIAL  SECURITY 

NUMBER  □ 

□ READ  LABELS  OR  INSTRUCTIONS  □ 

□ READ  / UNDERSTAND  NUTRITION 

INFORMATION  □ 

□ READ  A CALENDAR  □ 

□ READ  A BUS  SCHEDULE  □ 

□ READ  A T.V.  GUIDE  □ 

□ READ  A PHONE  BOOK  □ 

□ READ  MENUS  OR  RECIPES  □ 

□ READ  BILLS  □ 

□ READ  MAPS  □ 

□ READ  NEWSPAPERS  □ 

□ READ  MAGAZINES  □ 

□ READ  HEALTH  INFORMATION  □ 

□ READ  AND  WRITE  CHECKS  □ 
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□ READ  AND  WRITE  LETTERS  □ 

□ READ  AND  FILL  OUT  FORMS  □ 

□ READ  TO  CHILDREN  □ 

□ HELP  WITH  HOMEWORK  □ 

□ USE  A DICTIONARY  □ 

□ READ  NOTICES  FROM  SCHOOL  □ 

□ WRITE  NOTES  TO  SCHOOL  □ 

I KNOW  THIS  □ I WANT  TO  KNOW  MORE 
ABOUT  THIS 

GRAMMAR  SKILLS 

□ CAPITALIZATION  □ 

□ PUNCTUATION  □ 

□ SPELLING  □ 

□ DEFINITIONS  □ 

WRITING  SKILLS 

□ WRITE  IN  A JOURNAL  OR  DIARY  □ 

□ WRITE  SONGS  □ 

□ WRITE  POEMS  □ 

□ WRITE  FOR  A NEWSLETTER  □ 
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I WANT 

I KNOW  TO  KNOW 

THIS  MORE  ABOUT 

THIS 

□ WRITE  TO  A NEWSPAPER  □ 

□ WRITE  ADVICE  TO  OTHERS  □ 

□ WRITE  THE  GED  ESSAY  □ 

□ WRITE  WORK  OR  SCHOOL  REPORTS  □ 
LIFE  SKILLS 
□ REGISTER  TO  VOTE  □ 
lj  GET  A LIBRARY  CARD  □ 

□ GET  A LEARNER'S  PERMIT  □ 

LIFE  SKILLS 

□ UNDERSTAND  WANT  ADS  □ 

□ DO  A RESUME  □ 

□ FILL  OUT  A JOB  APPLICATION  □ 

□ UNDERSTAND  WORK  MANUALS  □ 

□ UNDERSTAND  WORKPLACE  BENEFITS  □ 

OTHER  GRAMMAR,  WRITING,  LIFE,  WORK 
SKILLS  I WOULD  LIKE  TO  KNOW  ARE: 


I WANT 

I KNOW  TO  KNOW 

THIS  MORE  ABOUT 

THIS 

□ DIVISION  □ 

□ WEIGHTS/MEASURES 
(SCALE,  THERMOMETER)  □ 

□ MEASUREMENT  - STANDARD  □ 

□ MEASUREMENT  - METRIC  □ 

□ TELLING  TIME  □ 

□ DECIMALS  □ 

□ FRACTIONS  □ 

□ RATIO  / PROPORTION  □ 

□ PERCENT  □ 

□ WORD  PROBLEMS  □ 

GED,  WORK,  COLLEGE  SKILLS 

□ BE  PREPARED  FOR  GED  WRITING/ 

GRAMMAR  TEST  □ 

□ BE  PREPARED  FOR  GED  WRITING/ 

ESSAY  TEST  □ 

□ BE  PREPARED  FOR  GED  ARTS 

& UTERATURE  TEST  □ 

□ BE  PREPARED  FOR  GED  SOCIAL  STUDIES 
TESTO 


MATH  SKILLS 

□ ADDITION  □ 

□ SUBTRACTION  □ 
□ MULTIPLICATION  □ 


□ BE  PREPARED  FOR  GED  SCIENCE  TEST  □ 
□ BE  PREPARED  FOR  GED  MATH  TEST  □ 

□ JOB  TRAINING  MATH  □ 

□ USE  A CALCULATOR  □ 

□ TYPE  □ 

□ USE  A COMPUTER  □ 
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I WANT 

I KNOW  TO  KNOW 

THIS  MORE  ABOUT 

THIS 

□ COLLEGE  WRITING  SKILLS  □ 

□ COLLEGE  MATH  SKILLS  □ 

□ COLLEGE  TEST  TAKING  SKILLS  □ 

OTHER  MATH,  GED,  WORK,  OR  COLLEGE 
SKILLS  I WOULD  LIKE  TO  KNOW  ARE: 


MATH/UFE  SKILLS 

□ UNDERSTAND  PAYCHECK 
(NET,  GROSS,  S.S.,  ETC.)  □ 

□ COMPARE  BENEFITS  □ 

□ SCHEDULE  WORK  HOURS  □ 
□ MAKE  A BUDGET  □ 

□ SET  SAVINGS  GOALS  □ 

□ UNIT  PRICE  FOOD  □ 

□ FIGURE  SALES  TAX  □ 


I WANT 

I KNOW  TO  KNOW 

THIS  MORE  ABOUT 

THIS 

□ FIGURE  RENTAL  COST  □ 

□ FIGURE  REPAIR  / REPLACEMENT  COST  □ 

□ FIND  DIMENSIONS  OF 
A SCALE  DRAWING  □ 

□ USE  PASSBOOK/CHECKBOOK  □ 

□ FIGURE  INTEREST  ON  SAVINGS 

OR  LOAN  □ 

OTHER 

□ USE  A MAP. ..LOCAL  □ 

□ STATE  □ 

□ u.s.  □ 

□ WORLD  □ 

OTHER  MATH  SKILLS  I WOULD  LIKE  TO 
KNOW  ARE: 


□ COMPARE  GENERIC 

& BRAND  NAME  ITEMS  □ OTHER 


□ FIGURE  CAR  EXPENSES  □ 

□ UNDERSTAND  A BILL 
(PHONE,  ELECTRIC,  HEAT)  □ 

□ HOW  TO  SAVE  ENERGY  □ 

□ FIGURE  AN  INSTALLMENT  PLAN  □ 
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I HAVE  I WOULD  LIKE 

READ  THIS  TO  READ 


□ ADVENTURE  □ 
□ ANIMALS  □ 

□ ARTO 

□ ASTROLOGY  □ 
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I HAVE  READ  I WOULD 

THIS  LIKE  TO  READ 

□ CHILDREN  □ 

□ COMEDY  □ 

□ COMIC  BOOKS  □ 

□ COOK  BOOKS  □ 

□ CULTURE  □ 

□ CURRENT  EVENTS  □ 

□ DRAMA  n 

□ ENTERTAINMENT  □ 

□ EXPERIMENTAL  □ 

□ FANTASY  □ 

□ FASHION  □ 

□ HEALTH  □ 

□ EMOTIONAL  □ 

□ FIRST  AID  □ 

□ HUMAN  BODY  □ 

□ NUTRITION  □ 

□ HISTORICAL  □ 

□ HOBBIES  (TYPE?)  □ 

□ HORROR  □ 

□ HUMOR  □ 

□ INSPIRATIONAL  □ 

□ INTELUGENCE  □ 

□ JOBS  □ 

□ JUVENILE  □ 

O 
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I HAVE  READ  I WOULD 

THIS  LIKE  TO  READ 

□ LEARNING  □ 

□ MOVIES  □ 

□ MUSIC  □ 

□ MYSTERY  □ 

□ NATURE  □ 

□ OCCULT  □ 

□ PARENTING  □ 

□ POETRY  □ 

□ POLITICAL  □ 

□ RECOVERY  □ 

□ REAL  PEOPLE  □ 
RELIGIOUS/SPIRITUAL 

□ MYSTICAL  □ 

□ ROMANCE  □ 

□ SELF-HELP  □ 

□ SCIENCE  □ 

□ SCIENCE  FICTION  □ 

□ SPORTS  □ 

□ SUSPENSE  □ 

□ TELEVISION  □ 

□ TRAGEDY  □ 

□ TRAVEL  □ 

□ WESTERNS  □ 
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How  Much  and  What  Kind? 

One  Family  literacy  Program's 
Assessment  Story 


by 

Sylvia  Greene, 
Nancy  Hoe,  and 
Lally  Stowell 


Community 
Learning  Center 
Cambridge,  MA 


The  topic  of  assessment  is  a fascinat- 
ing, challenging,  and  daunting  one 
for  any  adult  education  program, 
no  matter  what  its  scope  and  setting.  In  the 
field  of  family  literacy,  however,  assess- 
ment must  broaden  to  include  children  as 
well  as  parents,  and  curriculum  compo- 
nents such  as  parenting,  parent-and-child 
time  and  home  visiting  along  with  the  tra- 
ditional conversation,  reading,  grammar, 
writing  and  math. 

At  the  Cambridge  Even  Start  program, 
we  have  tried  to  deal  with  these  challenges 
by  doing  everything  on  the  assessment 
map:  standardized  and  non-standardized; 
group  and  individual;  formal  and  infor- 
mal; teacher  and  self.  You  name  it,  we  do 
it:  anecdotal  running  records,  work  sam- 
pling, conferences  and  I.E.P.S,  journals, 
locked  confidential  files,  open  accessible 
portfolio  files.  The  result  is,  as  our  local 
evaluator  put  it  recently,  we  are  "drown- 
ing in  data." 

If  assessment  is  always  a work  in 
progress,  the  above-mentioned  evaluator, 
Elizabeth  Brach,  is  the  latest  chapter  in  our 
assessment  story.  (Elizabeth  works  for  the 
Office  of  Resource  Development  and  As- 
sessment at  the  Cambridge  School  Depart- 
ment, directed  by  Barbara  Black.)  Every 
program  should  be  so  lucky  as  to  experi- 
ence an  outside  evaluator  as  positive, 
appreciative,  fair,  rigorous,  professional, 
and  willing  to  look  at  existing  assessment 
protocols  and  ask  hard  questions.  Our  lat- 
est revisions  and  attempts  to  streamline 
owe  a great  deal  to  her. 
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Cambridge  Even  Start  is  a collaboration 
between  the  Cambridge  Community 
Learning  Center  and  the  Cambridge 
School  Department  (primarily  its  Home- 
Based  Early  Childhood,  Primary  Education 
and  Title  I programs).  Families  attend  five 
mornings  a week  from  9 to  12  at  the  Gately 
Shelter  in  North  Cambridge,  a multi-use 
city  building  housing  youth  programs  in 
the  afternoon  and  evening.  Even  Start  is  a 
"center-based"  program  in  that  most  of  the 
components  happen  in  the  same  place  at 
the  same  time;  however,  each  family  also 
receives  a weekly  hour-long  home  visit  at 
a time  convenient  to  them  and  their  home 
visitor,  and  we  also  take  some  field  trips 
off-site.  We  are  designed  to  serve  30  Cam- 
bridge families  a year.  Since  our  beginning 
in  1993,  we  have  served  84  families  from 
20  different  countries  speaking  12  different 
languages,  reflecting  the  diversity  of  Cam- 
bridge. 

For  the  parents,  we  offer  an  ESL  class 
(Level  2/3  in  the  Community  Learning 
Center's  sequence)  taught  by  Lally  Stowell, 
and  an  ABE  class  (Intermediate/Pre-GED 
level)  taught  by  Sylvia  Greene,  on  Tues- 
days 9-12,  Thursdays  9-11,  and  Fridays 
9-11,  for  a total  of  seven  hours  a week. 
Monday  and  Wednesday  are  for  computer 
classes  (taught  by  Javier  Aponte),  with 
parents  coming  one  day  or  the  other  for 
three  hours.  On  those  days,  other  offerings 
are  an  hour-long  phonics  and  spelling 
class,  individual  tutoring,  group  home- 
work time,  and  time  for  parents  to  volun- 
teer in  the  preschool  class.  Fridays  from 
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11-12  we  have  a parent  discussion/sup- 
port group  for  ESL  and  ABE  parents  com- 
bined. Thursdays  from  11-12  we  have 
Parent  and  Child  Time  (PACT)  involving 
all  parents,  all  children,  and  all  staff.  Par- 
ent and  child  activities  centered  around  a 
book,  toy,  or  game  are  also  the  central  focus 
of  the  home  visits. 

Our  preschool  class  meets  every  morn- 
ing from  9-12,  and  is  open  to  children  2 
years  9 months  to  kindergarten  age.  It  is 
taught  by  Nancy  Hoe,  Estalina  Rodriguez, 
and  June  Ramdewar.  (We  provide 
babysitting  reimbursements  for  children 
under  2 years  9 months.)  About  half  the 
parents  have  their  children  in  our  pre- 
school class,  and  the  others  have  their 
children  in  Head  Start  or  other  preschool 
settings,  family.daycare,  or  elementary 
school.  We  run  from  September  to  the 
end  of  July,  and  are  open-entry,  open-exit. 

Even  Start  is  staffed  by  two  Co-Coordi- 
nators (Nancy  is  Early  Childhood  Co- 
Coordinator  and  Sylvia  is  ABE  Co-Coordi- 
nator) who  each  administer  half-time  and 
teach  half-time;  three  half-time  teachers 
(Lally  who  works  with  Sylvia,  and  Estalina 
and  June  who  work  with  Nancy);  a 10- 
hour-a-week  computer  teacher  (Javier); 
and  a Harvard  Graduate  School  of  Educa- 
tion intern  and  four  volunteer  tutors  who 
help  out  in  different  parts  of  the  program. 
Counseling  is  provided  by  the  teaching 
staff  and  agencies  to  whom  we  refer  par- 
ents as  needed. 

INITIAL  ASSESSMENT 

Parent 

New  families  come  to  us  in  several 
different  ways:  through  agency  referral, 
word-of-mouth,  cable  TV  announcements, 
targeted  AFDC  mailings,  and  so  on.  Over 
the  telephone,  it  is  usually  possible 
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through  evaluating  conversational  ability 
and  asking  a few  questions  about  previous 
education  to  determine  whether  the  parent 
is  more  appropriate  for  Lally's  ESL  class  or 
Sylvia's  ABE  class,  or  not  appropriate  for 
Even  Start  at  all,  in  which  case  we  try  to  re- 
fer the  person  elsewhere.  This  initial  phone 
call  is  the  first  step  in  the  assessment  process. 

Next  comes  an  appointment  with  Lally 
or  Sylvia,  in  the  parent's  home  or  at  the 
Gately  Shelter.  A Community  Learning 
Center  registration  form  is  filled  out,  yield- 
ing the  usual  demographic  data,  plus  edu- 
cational history  of  the  parent,  and  goals  the 
parent  chooses  for  themselves  from  the 
range  provided  on  the  form  (which  is  use- 
ful, but  not  complete).  A separate  Child 
Intake  Form  is  filled  out,  to  be  discussed 
later  in  this  article.  At  this  point,  the  ESL 
and  ABE  initial  assessments  diverge. 

ESL:  The  next  step  in  the  ESL  assessment 
is  a teacher-made  series  of  graded  readings 
designed  by  teachers  at  the  Community 
Learning  Center  that  the  parent  reads 
aloud,  with  Lally  asking  set  questions  after 
each  reading.  She  notes  numbers  of  hesita- 
tions and  mispronunciations,  and  level  of 
comprehension,  which  are  recorded  on  the 
form  and  filed  eventually  in  the  parent's 
folder.  Based  on  this  reading  inventory 
and  informal  conversation  with  the  parent, 
she  determines  if  he  or  she  is  appropriate 
for  her  class.  If  the  parent  scores  too  low, 
Lally  refers  him  or  her  to  another  class  at 
the  Community  Learning  Center  — ESL  1 
or  ESL  Literacy  1 (for  people  who  are  also 
not  literate  in  their  own  language)  — an- 
other adult  education  program,  or,  in  some 
cases,  loans  a series  of  her  own  tapes  to 
help  the  parent  reach  the  level  of  her  class. 
("Too  low"  would  mean  ESL  1 level:  some- 
one who  speaks  hardly  any  English  and 
would  not  be  able  to  sit  in  a group  of 
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Sylvia  analyzes 
the  results  with 
the  parent,  noting 
strengths  and 
needs  using  a 
checklist  of  skills 
ranging  from 
basic  computation 
skills  up  to 
geometry  and 
algebra. 


adults  from  many  different  countries  and 
have  a simple  conversation  about  a topic 
like  discipline  or  bedtime  routines.) 

There  is  no  second  form  of  this  informal 
reading  inventory  to  re-administer  later  in 
the  year  to  check  progress,  so  Lally,  along 
with  the  ESL  department  at  the  Commu- 
nity Learning  Center,  will  be  working  on 
designing  new  ESL  assessments  that  have 
two  forms,  one  for  pre-testing  and  one  for 
post-testing. 

ABE : For  ABE  parents,  Sylvia  next  uses 
two  interest  inventories:  one  is  general, 
asking  questions  about  the  parent's  work- 
related  reading,  writing  and  math  needs, 
topics  s/he  likes  to  read  about,  what  s/he 
does  well,  who  s/he  most  admires,  and 
where  s/he  would  most  like  to  travel.  The 
second  is  a more  specific  menu  of  possible 
GED-type  topics,  divided  into  the  areas  of 
Science,  Social  Studies  and  Literature, 
which  the  parent  can  rank  in  order  of  inter- 
est. The  topics  are  ones  like  the  civil  rights 
movement,  poetry,  stress  reduction,  geog- 
raphy of  the  Caribbean,  and  so  on,  that 
have  been  of  interest  in  the  past  to  many 
adult  learners,  not  just  GED  students.  (At 
Even  Start,  a majority  of  the  ABE  students 
are  interested  in  getting  a GED.  The  others 
want  to  improve  their  reading,  writing  and 
math,  usually  with  job  training  or  job  ad- 
vancement in  mind.) 

Next,  the  parent  does  a writing  sample, 
and  then  with  Sylvia  analyzes  it  for 
strengths  and  needs  using  a checklist  of 
writing  components,  including  handwrit- 
ing, mechanics,  spelling,  vocabulary,  use  of 
standard  grammar,  ability  to  stick  to  the 
topic,  ability  to  combine  thoughts  and  feel- 
ings, and  so  on.  This  is  kept  in  the  parent's 
portfolio  folder,  for  diagnostic  purposes 
and  for  comparison  with  later  writing 
samples  chosen  by  the  parent  to  measure 
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progress.  The  Community  Learning  Center 
staff  is  presently  in  the  process  of  develop- 
ing a "writing  rubric"  to  facilitate  wholistic 
scoring  of  writing  samples,  and  Even  Start 
looks  forward  to  benefiting  from  this  effort. 

Usually,  this  is  enough  for  the  first 
meeting,  as  1-1/2  to  2 hours  have  often 
passed.  In  the  second  session,  the  parent  is 
given  the  Diagnostic  Assessments  of  Read- 
ing, or  DAR  (Florence  G.  Roswell  and 
Jeanne  S.  Chall,  The  Riverside  Publishing 
Company,  8420  Bryn  Mawr  Ave.,  Chicago, 
IL  60631).  This  standardized  but  informal, 
individually-administered  test  yields 
grade  levels  and  diagnostic  information  in 
five  reading  components:  isolated  word 
recognition,  word  analysis,  word  recogni- 
tion in  context,  oral  vocabulary,  and  silent 
reading  comprehension,  as  well  as  spell- 
ing. A relatively  short,  flexible,  easy-to- 
administer  instrument,  the  DAR  results 
in  a profile  extremely  useful  in  designing 
instruction  specific  to  the  parent's  needs. 
The  instrument  was  chosen  by  the  ABE 
Department  at  the  Community  Learning 
Center  as  most  useful  for  looking  at  several 
components  of  reading. 

We  also  give  the  TABE  (Tests  of  Adult 
Basic  Education,  CTB/McGraw  Hill,  20 
Ryan  Ranch  Rd.,  Monterey,  CA  93940)  in 
order  to  satisfy  our  funders,  be  consistent 
with  the  Community  Learning  Center,  and 
provide  transition  to  job  training  centers 
that  use  TABE  scores.  We  give  the  TABE 
Locator,  then  the  indicated  level  of  the  test. 
(We  give  only  the  Reading  subtest.) 

Next  comes  the  Community  Learning 
Center's  informal  math  diagnostic.  As  with 
the  writing  sample,  Sylvia  analyzes  the  re- 
sults with  the  parent,  noting  strengths  and 
needs  using  a checklist  of  skills  ranging 
from  basic  computation  skills  up  to  geom- 
etry and  algebra. 
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The  DAR,  TABE  and  math  diagnostic 
tests  are  all  filed  in  the  parent's  locked, 
confidential  file.  This  is  plenty  for  the  sec- 
ond session.  At  this  point  the  parent  can 
begin  class. 

The  last  step  in  initial  ABE  assessment 
happens  after  s/he  has  been  in  class  a 
week  or  so.  The  Literacy  and  Numeracy 
Practices  Questionnaire  is  a fancy  name  for 
a list  of  real-life  reading,  writing,  and  math 
survival  skills,  grouped  into  those  used  at 
home,  at  work,  and  out  in  the  community. 
They  include  such  things  as  reading  the 
newspaper,  figuring  tips,  writing  a resume 
and  filling  out  an  accident  report.  For  each 
skill  there  are  columns  for  "Can  do"  al- 
ready (and  if  so,  "How  often?"  and  "Easy 
or  Hard?"),  and  "Would  like  to  learn." 

Based  on  the  preceding  information 
from  all  the  assessments  done  so  far,  it  is 
now  possible  to  sit  down  with  the  parent 
and  have  an  initial  conference  to  set  goals. 
Goals  can  be  chosen  from  the  "Needs 
work"  column  of  the  checklists  for  the 
writing  sample  and  math  diagnostic,  from 
the  "Would  like  to  learn"  column  of  the 
questionnaire,  or  the  topics  ranked  high  on 
the  interest  inventories.  The  form  used  for 
the  conference  in  the  ABE  class  (a  different 
one  is  used  in  the  ESL  class)  is  on  the  next 
page. 

All  this  assessment,  then,  leads  to  the 
parent's  conference  record,  which  informs 
both  group  and  individual  instruction  and 
helps  track  progress.  When  a parent  com- 
pletes one  of  their  short-term  goals  as 
stated  on  this  form,  he  or  she  receives  a 
certificate,  a copy  of  which  may  go  in  their 
portfolio  file. 


Child 

During  the  first  meeting  with  the  parent, 
a Child  Intake  Form  is  filled  out  for  each 
child  in  the  family  aged  seven  or  younger. 
This  yields  birthdate;  parents'  names; 
names,  ages  and  school  placement  of  all 
siblings;  favorite  activities;  allergies;  other 
services  and  agencies  involved  with  the 
child;  and  the  parents'  goals  for  the  child. 

During  the  first  home  visit  to  the  family, 
Nancy  Hoe  or  the  Home-Based  Director 
(Ellen  Grant  Valade)  takes  a developmen- 
tal history  for  each  child  aged  seven  or 
younger.  They  also  give  the  Peabody 
Picture  Vocabulary  Test  (PPVT),  a well- 
respected  standardized  instrument  which 
tests  the  child's  receptive  vocabulary  and 
concept  development. 

Family 

Soon  after  parents  begin  class,  they  are 
asked  their  goals  for  the  whole  family. 
These  are  often  difficult  to  articulate,  and 
require  discussion  and  modeling  in  the  Fri- 
day parent  discussion  group.  Parents  often 
take  time  to  articulate  their  goals,  so  a list 
of  examples  from  other  parents  is  often 
helpful.  (An  example  of  a family  goal  is  to 
spend  more  time  together  on  the  week- 
ends.) Their  goals  are  filed  in  their  family 
portfolio  folder.  For  each  family,  there  are 
two  sets  of  fairly  comparable  files,  one 
kept  inside  a locked  cabinet,  and  one  in  a 
portable  milk  carton  on  top  of  the  cabinet 
accessible  to  parents  and  teachers.  In  each 
place,  there  is  a hanging  file  for  each  fam- 
ily containing  four  differently-colored  fold- 
ers, one  for  the  parent's  adult  class,  one  for 
each  child  seven  or  younger,  one  for  home 
visiting  and  one  for  the  family. 


Soon  after  parents 
begin  class,  they 
are  asked  their 
goals  for  the  whole 
family.  These  are 
often  difficult  to 
articulate,  and 
require  discussion 
and  modeling  in 
the  Friday  parent 
discussion  group. 
Parents  often  take 
time  to  articulate 
their  goals,  so  a 
list  of  examples 
from  other  parents 
is  often  helpful. 
(An  example  of  a 
family  goal  is  to 
spend  more  time 
together  on  the 
weekends.) 
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CAMBRIDGE  EVEN  START  Name  _ 

INDIVIDUAL  EDUCATIONAL  PLAN  Date 

and  CONFERENCE  RECORD  FORM  Class  _ 

Teacher 


1 . 1 think  my  reading  is 

I think  my  writing  is 

I think  my  spelling  is 

I think  my  math  is 

I think  my  parenting  (helping  my  children  develop  in  a positive  way)  is 


2.  What  goals  would  you  like  to  work  on  in  the  next  5 months? 


Goal 


Date 

chosen 


Date 

accomplished 


Reading: 


Writing: 


Spelling: 


Math: 


Parenting: 
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3.  Teacher's  goals  for  the  student  for  the  next  5 months: 

Reading: 

Writing: 

Spelling: 

Math: 

(over) 

Teacher  assessment 

Self-assessment 

4.  Comments  oboul  un6ndunC3! 

5.  Comments  about  homework: 

6.  Previous  test  score: 

(name  of  test)  (date  of  test) 

(score) 

Previous  test  score: 

(name  of  test)  (date  of  test) 

(score)  (score) 

-• 

6/97 
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ONGOING  ASSESSMENT 

Family 

The  most  obvious  ongoing  assessment 
information  is  attendance  records.  We 
were  keeping  separate  attendance  forms 
for  parents,  children  in  our  preschool  class, 
and  home  visits;  now  we  are  consolidating 
these  into  one  family  attendance  form  to  be 
filed  at  the  end  of  each  month  in  the  locked 
family  file.  Also,  family  goals  are  updated 
every  five  months  during  individual  con- 
ferences or  in  a parent  discussion  group. 

Parent 

Another  obvious  source  of  assessment 
data  is  a record  of  homework  assigned  and 
either  completed  or  not  completed.  Discus- 
sion of  both  attendance  and  homework 
records  is  included  in  the  conference.  Con- 
ferences happen  in  January  and  June  and 
include  notation  of  goals  met  and  setting 
of  new  goals. 

The  TABE  is  re-administered  to  ABE 
parents  right  before  these  conferences  and 
discussion  of  (hopefully!)  progress  as 
shown  by  TABE  scores  is  included.  Lally 
uses  a group  of  five  parenting  issues  (such 
as  discipline  and  giving  children  responsi- 
bility) for  ongoing  assessments  every  two 
months  during  the  year.  She  pairs  two  ESL 
students  to  discuss  the  topic,  then  has  them 
separate  and  write  on  the  topic;  discussion 
of  these  writing  samples  is  part  of  her  indi- 
vidual conferences. 

ABE  parents  keep  journals  which  they 
write  in  each  Friday,  answering  two  ques- 
tions: "What  did  you  learn  in  your  adult 
class  today?"  and  "Give  an  example  of 
something  you  did  this  past  week  to  help 
your  child  learn,"  the  second  question  hav- 
ing been  suggested  by  our  first  local  evalu- 
ator as  a positive,  non-judgemental  and 
open-ended  way  for  parents  to  think  about 
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It  became  clear 
that  there  was  no 
opportunity  for 
the  parent  to 
evaluate  all  five 
components  of 
Even  Start  at 
once,  thus 
reflecting  the 
comprehensiveness 
of  family  literacy. 


their  parenting. 

Copies  of  all  writing  samples  are  kept 
in  the  parent's  accessible  portfolio  file,  and 
at  conferences  can  be.compared  using  the 
same  checklist  used  for  the  initial  writing 
sample  to  show  changes. 

Since  math  instruction  in  the  ABE  class 
is  by  necessity  individualized,  each  stu- 
dent has  a running  Individualized  Math 
Record  Sheet,  with  columns  for  book,  page 
number,  concept  worked  on,  date  given, 
and  date  received.  These  are  kept  in  a 
folder  in  the  classroom. 

For  each  parent,  Sylvia  also  keeps  a run- 
ning Teacher/Tutor  Comment  Sheet, 
where  she  or  one  of  the  volunteer  tutors 
who  help  in  the  classroom  can  note  any- 
thing important.  These  sheets  are  confiden- 
tial and  are  kept  in  a folder,  eventually 
filed  in  the  parent's  folder  in  the  locked 
cabinet. 

Recently,  thanks  to  feedback  from  our 
local  evaluator  Elizabeth  Brach,  it  became 
clear  that  there  was  no  opportunity  for  the 
parent  to  evaluate  all  five  components  of 
Eyen  Start  at  once,  thus  reflecting  the  com- 
prehensiveness of  family  literacy.  There- 
fore, we  came  up  with  the  following  form, 
to  be  filled  out  by  the  parents  every  other 
Friday  during  Parent  Discussion  time.  The 
form  has  been  modified  several  times,  and 
the  final  product  owes  much  to  the  valu- 
able feedback  of  Home-Based  Director 
Ellen  Grant  Valade.  We  have  tried  it  once, 
and  hope  it  will  help  us  keep  an  eye  on  the 
program  as  a whole.  After  the  parent  has 
filled  it  out,  the  form  will  be  filed  in  the 
locked  family  file. 
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"THIS  WEEK  'l~.'  FORM" 

NAME Week  ending  Friday, , 1 99_ 

SELF-ASSESSMENT:  "This  past  two  weeks,  I and/or  my  child  attended  . . . 


ACTIVITY/COMPONENT 

Yes 

No 

Comments 

Adult  class: 

ESL  or  ABE 

Computer  class 

Phonics/ Spelling 

Individual  tutoring 

Parent  discussion 

Contact  w.  child's  school 

(Comment  on  any  talks  you,  the  parent, 
had  with  the  teacher,  volunteering  in 
the  classroom,  etc.) 

Even  Start  preschool 

Headstart,  CEOC, 
other  preschool 

Family  daycare/babysitter 

Elementary  school 

Home  Visit 

Parent-and-child  time  or  field  trip 
Reading  to  my  children 
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"THIS  WEEK  I..."  FORM 

The  history  of  the  form  on  the  previous 
page  is  an  example  of  the  ongoing  adapta- 
tion of  our  assessment  practices.  We  used 
to  have  a separate  home  visit  report  form, 
and  a separate  school  contact  report  form 
and  a separate  form  to  report  books  par- 
ents read  to  children  during  the  week. 
Now  the  school  contact  and  home  visit 
forms  are  absorbed  into  the  above  one  and 
thanks  to  a suggestion  from  Ellen  Grant 
Valade,  the  home  visitor  will  now  help 
the  parent  keep  a running  record  of  books 
read,  since  it  will  be  easier  to  do  in  the 
home  where  the  books  are  right  there  in 
plain  sight. 

Each  July,  during  home  visits,  Sylvia 
fills  out  a form  with  parents  asking  them 
to  comment  on  their  own  progress,  their 
children's  progress,  and  their  plans  for 
September. 

Child 

Work  samples  are  collected  throughout 
the  year  for  each  child  in  the  Even  Start 
preschool  class,  and  extensive  running  an- 
ecdotal records  are  kept.  A developmental 
checklist  is  filled  out  for  the  child  at  mid- 
year, then  again  at  the  end  of  the  year. 
(Nancy  is  presently  working  on  a specific 
literacy  development  checklist  covering 
birth  to  7 years  linked  to  an  instrument  the 
Cambridge  School  Department  is  begin- 
ning to  use  which  is  based  on  Marie  Clay's 
Concepts  About  Print,  or  CAP.)  Preschool 
staff  write  short  monthly  goals. 

In  July,  an  End-of-the-Year  Report  is 
written  for  each  child  using  the  Cambridge 
School  Department's  Kindergarten  Transi- 
tion Form.  A copy  of  this  report  is  given  to 
the  parent.  This  coming  year,  we  will  be- 
gin keeping  a parent-child  spiral  notebook 
for  each  child  in  the  preschool  class,  in 
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which  staff  can  write  down  information 
about  the  child,  and  where  parents  can 
read  and  respond.  These  will  be  kept  in 
an  accessible  box  in  the  preschool  room. 
Finally,  the  Peabody  Picture  Vocabulary 
Test  is  readministered  by  Nancy  during 
the  July  home  visit. 

EXIT  ASSESSMENT 

Since  parents  rarely  leave  Even  Start 
with  plenty  of  notice,  this  aspect  of  our 
assessment  protocol  is  weakest.  Ideally,  we 
hope  to  give  ABE  parents  a final  TABE,  all 
children  a final  PPVT,  and  all  parents  our 
Exit  Interview,  which  includes  questions 
about  their  future  plans  and  their  ratings 
of  the  different  components  of  the  program. 

After  all  is  said  and  done,  what  is  the 
purpose  of  all  this  data.collection?  Within 
our  program,  it  clearly  helps  parents  docu- 
ment and  celebrate  their  own  and  their 
children's  progress.  Even  Start  also  needs 
to  make  sure,  however,  that  their  assess- 
ment practices  are  meaningful  to  potential 
funders,  since  our  federal  grant  runs  out 
the  year  2001.  We  need  to  demonstrate 
that  we  are  doing  some  good  in  terms 
that  make  sense  to  taxpayers  and  school 
departments  and  state  legislators  and  who- 
ever else  might  be  interested  in  eventually 
adopting  us. 

For  this  reason,  the  eleven  Even  Start 
programs  around  Massachusetts  - already 
accustomed  to  working  together,  thanks  to 
a caring  and  energetic  statewide  coordina- 
tor at  D.O.E.  (Arlene  Dale)  who  assembles 
us  regularly  — have  begun  to  discuss  the 
idea  of  looking  at  family  literacy  assess- 
ment from  a statewide  perspective. 

In  addition  to  the  local  and  state  layers 
of  assessment,  there  is  a national  layer. 

Like  the  other  600  or  so  Even  Start  programs 
around  the  country,  we  have  to  report  each 
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July  to  the  Even  Start  Information  System 
(ESIS)  standard  demographic  and  atten- 
dance data,  among  other  things.  For  two 
years,  we  were  also  part  of  something 
called  the  National  Sample  study,  and  for 
a limited  number  of  our  families,  reported 
TABE  or  CASAS  scores  for  parents  and 
Preschool  Language  Survey  scores  for  their 
preschool  children. 

The  resulting  National  Even  Start  Evalu- 
ation was  disappointing  because  it  did  not 
capture  at  all  the  daily  progress  practitio- 
ners were  seeing.  Partly  in  response  to  this 
situation,  a newly-formed  national  organi- 
zation of  practitioners  called  the  National 
Even  Start  Association  is  moving  toward 
doing  its  own  evaluation,  which  will  hope- 
fully document  this  progress.  Nancy  is  on 
their  Advisory  Board. 


Our  assessment  practices  seem  to  re- 
quire constant  tinkering,  and  we  find  our- 
selves trying  this  and  rejecting  that,  seeing 
what  works  and  doesn't  work,  and  listen- 
ing, reading,  sharing,  and  certainly  bor- 
rowing. Please  feel  free  to  adapt  any  of  the 
instruments  mentioned  in  this  article,  or 
contact  us  with  any  questions  or  ideas. 
(Sylvia  Greene  and  Lally  Stowell  can  be 
reached  at  the  Community  Learning  Cen- 
ter, 19  Brookline  St.,  Cambridge,  MA 
02139,  tel.  617-349-6363;  Nancy  Hoe  is  at 
the  Cambridge  School  Department,  159 
Thorndike  St.,  Cambridge,  MA  02141,  tel. 
617-349-6493.)  Cambridge  Even  Start  looks 
forward  to  feedback  to  this  article,  and  to 
the  continuing  dialogue. 
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What  We  Had  to  Think  About  Before 
We  Could  Do  Portfolio  Assessment 


by 

Kathy  Sikes 


Durham  Literacy 
Council 
Durham,  NC 


The  Durham  Literacy  Council  is  a 
community  agency  with  a partici- 
patory focus.  Staff  members  and 
volunteer  tutors  work  in  a variety  of  set- 
tings from  community  centers,  university 
worksite  programs,  residential  substance 
abuse  centers,  jails  and  employment  train- 
ing programs.  Volunteer  training  focuses 
on  developing  lessons  from  learners  goals 
and  authentic  materials,  and  learners  work 
either  in  small  group  settings,  one  to  one 
tutoring  sessions  or  both. 

The  Lila  Wallace/Literacy  South  Portfo- 
lio Project  inspired  staff  members  to  exam- 
ine the  role  of  assessment  in  a learner 
centered  practice  and  to  create  the  pro- 
gram structure  necessary  for  implementa- 
tion. The  importance  of  participatory 
planning  and  ongoing  reflection  became  a 
major  focus  as  authentic  assessment  prac- 
tices were  explored.  I am  the  program  di- 
rector and  have  worked  for  the  Literacy 
Council  for  seven  years. 

INTRODUCTION 

The  portfolio  project  has  helped  us  at  the 
Literacy  council  think  about  a way  to  inte- 
grate assessment  and  instruction.  If  you're 
going  to  start  into  a portfolio,  then  you 
have  to  have  some  plan  in  the  beginning 
about  what  you're  after  in  your  learning, 
beyond,  "I  want  to  learn  to  read  and  write 
better"  or  "I  want  to  get  a GED  or  get  a 
drivers  license."  Tutors  and  students  are 
having  to  do  a lot  more  planning  together 
initially.  Even  though  we've  always  said 
we  were  participatory,  and  I think  we've 


done  the  best  job  we  could  do  with  that, 
sometimes  tutors  kind  of  fumbled  with 
focusing  on  students'  goals.  I think  it  was 
difficult  for  them  to  always  see  the  prog- 
ress they  were  making  towards  those 
goals,  especially  if  someone  didn't  read 
and  write  very  well  and  they  wanted  to 
get  a GED  or  something.  Doing  the  kind 
of  planning  that  makes  sense  and  there's 
something  there  to  attach  it  to,  it's  almost 
like  some  kind  of  mutual  contract,  mutual 
decision  making. 

We're  not  where  we  would  like  to  be  in 
terms  of  full  program  participation.  It  was 
difficult  for  us  to  get  started  and  I'm  not 
sure  why.  I think  we  really  struggled  with 
how  we  train  volunteers  about  portfolio, 
and  we're  still  struggling  with  that  some- 
what. The  piece  that  seemed  to  be  missing 
when  we  did  the  portfolio  training  was  the 
planning  piece.  Even  our  small  group  lead- 
ers were  saying  "We've  done  so  many 
kinds  of  things,  and  there  is  nothing  that 
really  ties  it  together."  I realize  that  they 
weren't  planning  very  well  and  couldn't 
do  portfolios  if  they  weren't  doing  better 
planning.  So  we're  just  now  getting  to  the 
point  that  we're  going  to  start  seeing 
people  have  real  portfolios  to  talk  about. 

INTRODUCING  PORTFOLIOS 

Our  teachers,  other  than  the  other  two 
people  on  our  staff,  are  all  volunteers  and 
so  part  of  my  charge  was  to  go  back  and 
train  and  disseminate  information  from 
the  portfolio  assessment  project.  I think  we 
did  a good  job  with  that.  We  held  different 
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workshops  - on  Saturday  morning,  one  in 
the  evening.  Early  on  the  workshops  were 
an  introduction  to  the  whole  portfolio  pro- 
cess because  we  were  really  new  at  it  too. 
We  just  wanted  to  get  started  and  we 
handed  out  some  things  like  cover  sheets 
and  portfolios,  and  said,  "Well,  go  and  try 
it  and  we'll  meet  in  a couple  of  months 
and  see  what  your  questions  .are."  That 
couple  of  months  passed  and  not  very 
many  people  did  anything.  Then  we  evalu- 
ated how  we  would  do  the  training  differ- 
ently. 

Next,  we  invited  volunteers  and  learn- 
ers who  were  trying  out  portfolio  assess- 
ment to  come  and  talk  with  other  tutors 
about  it.  It  wasn't  perfect  in  the  beginning. 
People  just  looked  at  the  portfolios  and 
talked  about  how  they  chose  things  or  why 
a certain  piece  was  in  there.  We  decided 
we'd  provide  ample  opportunity  for 
people  who  had  been  in  the  program  for 
a long  time  to  come  and  get  information 
about  portfolio  assessment.  We  wrote 
about  it  in  newsletters,  we  did  everything 
we  could  to  incorporate  existing  tutors. 
Then  we  decided  to  put  it  in  the  training. 
Now,  it's  part  of  the  tutor  training  that 
we  do  rather  than  an  in-service  workshop. 
Like  pre-service  training  rather  than  in- 
service  and  staff  development. 

When  we  planned  the  first  portfolio 
training  workshop,  we  tried  to  do  way  too 
much.  In  some  ways,  I think  we've  made 
this  an  overcomplicated  thing.  When  we 
first  started  having  conversations  in  the 
portfolio  assessment  project  about  portfo- 
lios, there  was  a lot  that  I didn't  under- 
stand. I did  not  understand  the  concept  of 
"criteria."  I'm  not  sure  why  not.  Some- 
times what  I want  to  learn  comes  to  me  in 
the  middle  of  things.  It  seemed  like  "crite- 
ria" was  an  esoteric  thing.  I didn't  know 
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the  process  well  enough  to  feel  like  I could 
change  it,  or  we  could  add  other  things.  I 
know  at  one  point  I felt  very  limited  about 
what  I had  chosen  for  my  criteria.  And 
then  I got  over  it.  I realized,  this  is  mine! 

I can  change  this.  It's  fine,  right?  That  part 
took  a long  time. 

We  struggled  with  how  to  talk  to  tutors 
about  portfolio  assessment,  and  basically 
what  we  said  was,  "this  is  a good  thing. 
Look  at  all  that  can  be  done  by  using  port- 
folios." We  gave  tutors  some  sample  les- 
sons reflecting  on  change.  If  I had  to  do  it 
over  again,  I would  just  skip  all  these  dis- 
cussions about  change  and  start  talking 
about  "how  do  you  know  you've  learned 
something?"  rather  than  it  being  some 
kind  of  introduction  into  "Now  we're  go- 
ing to  prepare  you  to  participate  in  this 
process."  I don't  know,  I think  I was  a little 
too  careful,  honestly,  and  not  with  learners 
with  tutors.  Then  finally,  I realized  that 
part  of  it  was  that  there  was  no  organiza- 
tional structure  that  supported  portfolio 
assessment  other  than  just  a philosophical 
commitment. 

Our  tutors  weren't  having  ongoing 
discussions  about  progress  with  their  stu- 
dents, and  they  didn't  have  anything  to 
base  it  on  either.  All  they  knew  was  some 
kind  of  large  goal  that  this  person  had  or 
didn't  have.  I hesitated  to  provide  models 
because  there  I didn't  want  teachers  and 
tutors  to  name  the  learning  for  the  learner. 
I was  concerned  that  we  could  interfere 
with  the  learners'  right  to  choose  and 
describe  their  learning  any  way  that  they 
wanted. 

DOING  PORTFOLIOS 

As  far  as  the  impact  on  students,  that's 
hard  to  know  yet.  I think  that  the  students 
will  increase  their  desire  to  be  more  par- 

7W7, 


Finally,  I realized 
that  part  of  it  was 
that  there  was  no 
organizational 
structure  that 
supported 
portfolio 

assessment  other 
than  just  a 
philosophical 
commitment. 

Our  tutors 
weren't  having 
ongoing 

discussions  about 
progress  with 
their  students, 
and  they  didn't 
have  anything  to 
base  it  on  either. 


ADVENTURES  IN  ASSESSMENT 


Volume 
10  • 56 


I tried  to  watch 
my  progress  and 
my  learning  and 
thinking  about 
original  criteria 
specifically.  But, 
not  everything 
I've  included  in 
my  portfolio  is 
about  that.  For 
example,  I 
included  part  of 
what  I wrote  for 
a small-group 
inquiry  based 
workshop, 
because  I made 
an  important 
connection. 


ticipatory  because  tutors  and  students  are 
supposed  to  work  together  to  plan  their 
work  for  the  quarter.  People  are  sending  us 
more  complete  plans.  And  tutors  and  stu- 
dents are  filling  them  out  together  to  plan 
their  work  for  the  quarter.  I see  two  differ- 
ent people's  handwriting  on  things.  So  I 
think  that  part  will  benefit  the  students. 

Students  will  also  benefit  from  looking 
at  their  work.  In  our  first  meeting,  I say 
that  it  became  really  clear  to  people  what 
they  hadn't  done.  I mean.  They  were  really 
proud  of  the  work  that  they  had  done,  but 
they  kept  seeing  these  gaps  of  things  that 
were  goals  they  had  not  met.  We  had  two 
people  who  said  "I  don't  have  any  writing 
to  put  in  this,"  and  "what  happened  to  that 
writing  class  we  used  to  come  to?"  And  I 
said,  "you  stopped  coming,  and  we  closed 
the  class.  You  know,  we  can't  just  run  it 
forever  with  one  person  in  it,  or  that  needs 
to  be  a one-to-one  match.  Oh,  well  maybe 
we  should  do  that  again." 

There  were  students  that  came  to  that 
meeting  that  only  worked  in  workbooks. 
They  had  much  less  to  put  in  their  portfo- 
lios; for  example,  they  saw  people  putting 
in  their  portfolios  evidence  that  they  had 
registered  to  vote  or  had  been  to  confer- 
ences and  had  collected  lots  of  material. 

We  had  talked  about  meeting  with  stu- 
dents individually  to  look  at  their  portfo- 
lios. That  evening,  four  students  out  of 
twelve  asked  me,  "when  are  we  going  to 
do  that?"  So  they  felt  invested  in  the  project. 

I'm  looking  forward  to  when  we  have  or 
next  meeting.  We'll  have  people  who  are  a 
little  further  along  in  looking  at  their  port- 
folios than  others.  But  my  guess  is  that 
rather  than  everybody  taking  the  same 
amount  of  time  to  get  to  that  point,  people 
will  get  there  faster  by  seeing  the  examples 
of  other  people  doing  it. 
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I've  loved  doing  my  own  portfolio.  I've 
Loved  it.  Particularly  the  last  time  that  we 
all  got  together  in  the  portfolio  project  and 
I got  to  spend  some  time  really  looking  at 
my  folder.  Until  then,  I just  collected 
things  really.  The  other  times  that  we  had 
to  sort  things  ,1  put  a few  things  in  my 
portfolio,  but  this  time  I really  knew  what  I 
wanted  in  it.  I also  didn't  edit  when  I was 
bringing  things.  I brought  a box  full  of  dif- 
ferent kinds  of  reading,  writings  and  work. 

Writing  the  cover  sheet  has  been  really 
informative  for  me.  The  first  thing  that  I 
think  was  important  to  mention  is  that  it's 
not  so  much  what  I write  on  the  cover 
sheet.  It  is  the  process  of  doing  it.  I mean, 
I've  written  handouts  for  years.  I've  writ- 
ten training  workshops.  I've  never  gone 
back  and  said,  "why  do  I like  this?  Why 
am  I keeping  this  now?"  It's  made  me  feel 
really  good  about  the  kinds  of  things  that 
I've  accomplished  in  my  thinking  and  my 
job. 

I tried  to  watch  my  progress  and  my 
learning  and  thinking  about  original  crite- 
ria specifically.  But,  not  everything  I've  in- 
cluded in  my  portfolio  is  about  that.  For 
example,  I included  part  of  what  I wrote 
for  a small-group  inquiry  based  workshop, 
because  I made  an  important  connection.  I 
realized  how  the  planning  part  was  miss- 
ing for  the  tutors.  They  were  saying  "We 
love  the  idea  of  portfolios,  because  we 
weren't  sure  whether  people  were  making 
progress,  and  it's  a hard  time  getting 
learner  input  about  what  to  do,  and  I 
thought,  we're  not  very  good  at  that,  but 
we  are  getting  somewhere  now." 

STAFF  DEVELOPMENT 

Portfolio  assessment  has  made  us  look  at 
our  training.  Volunteers  are,  in  essence, 
our  staff  and  as  such  we  expect  some 
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accountability  for  what's  happening  with 
their  work.  In  some  ways  I which  we  had 
a 20  hour  a week  person  who  just  paid 
attention  to  helping  people  plan,  being  a 
resource  to  them  in  their  planning  and 
then  visiting  them  about  their  portfolios. 
We  don't  provide  a lot  of  leadership  on 
how  to  get  input,  how  to  ask  good  ques- 
tions, and  how  to  plan  together.  We  just 
say  "Do  it.  Good  luck.  And  bring  back  a 
beautiful  portfolio."  So  it  was  the  connec- 
tion between  those  two  things,  planning 
and  ongoing  assessment,  that  made  me  re- 
alize where  the  training  needed  to  be  done. 

Right  now  in  training,  we  do  a three- 
hour  session  on  assessment,  mostly  talking 
about  what  happens  when  people  come  in 
the  door,  what  kind  of  initial  assessment. 
Then  we  talk  about  what  we  do  with  that 
information,  what  we  tell  the  tutor.  We 
talk  about  the  planning  sheets  and  how  the 
tutor  can  take  the  information  given  them 
and  then  have  this  more  in  depth  conver- 
sation with  the  learner.  Part  of  it's  for  your 
information  lecture  type  stuff,  but  then 
usually  at  some  point  we  say,  "OK,  now 
here's  some  information  we  would  have 
given  you  on  the  phone  about  learners," 
and  have  them  create  lessons  based  on  it, 
so  they  see  the  connection  between  the 
assessment  piece  and  the  lesson  piece. 
People  do  quite  a good  job.  I mean.  I've 
been  impressed.  We  don't  do  very  much 
on  what  happens  between  the  planning 
and  evaluating  how  well  it  went,  and 
that's  the  sort  of  thing  that  we  have  to 
work  on  in  our  training.  Then  we  say . 

"OK,  this  was  our  goal,  what  might  you 
put  in  a portfolio?  What  would  be  the 
range  of  things?"  All  we  can  do  at  this 
point  is  brainstorm  that.  We  don't  have 
anything  to  real  to  show  people.  We  cre- 
ated sample  portfolios  that  are  very  differ- 


ent from  each  other  and  show  them  to 
people  in  training.  That  seems  to  help.  It 
wouldn't  be  anything  like  if  learners  were 
coming  and  talking  about  their  portfolios, 
and  I think  we  can  pull  that  off  by  the  next 
training.  I do. 

One  of  the  other  activities  volunteers  do 
in  training  is  to  help  us  create  a training 
portfolio.  They  answer  two  questions  and 
then  talk  about  what  they  would  give  as 
evidence.  We  ask  questions  like,  what  did 
you  learn  about  teaching  and  learning  that 
you  didn't  know  before?  What  did  you  like 
most  about  the  training?  We're  getting 
much  better  feedback  about  what  people 
are  getting  out  of  training. 

CONCLUSION 

. My  colleague  Lee  and  I do  all  the  training, 
so  much  of  it  is  still  in  our  heads.  If  we  can 
create  some  kind  of  organizational  memory 
of  this  project  by  keeping  learners  doing 
it — and  by  January  we  should  have  people 
who  have  pretty  good  starts  on  good  port- 
folios— we  will  have  started  that  process.  If 
we  can  keep  that  going  and  have  students 
help  us  lead  portfolio  workshops  for  other 
learners  and  volunteers  then  in  a couple  of 
years  this  will  be  our  primary  assessment 
tool. 

It  has  been  a real  process  for  the  organi- 
zation, and  if  it  took  us  two  years  to  feel 
confident  about  it,  I think  we  need  to  give 
ourselves  that  much  time.  Initially,  we 
thought,  "this  will  be  a great  thing  to  do 
because,  one,  this  is  the  way  we  think  any- 
way and  learners  will  have  that  much 
more  ownership  over  another  piece  of 
what  they're  involved  with  at  the  literacy 
council,  and  it'll  be  great  to  show  the 
funders.  We'll  have  all  these  really  tangible 
things  for  people  to  see."  Actually,  the  big- 
gest impact  has  been  an  organizing  effect 
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— making  us  take  a real  look  at  training 
and  saying  — "what  are  we  saying  about 
assessment?  How  can  we  get  people  to  buy 
into  the  process?  How  can  we  get  them  to 
feel  like  it's  not  some  kind  of  esoteric 
thing,  or  additional  work,  but  make  it  as 


real  as  we've  made  writing  and  reading 
strategies?"  That's  were  we  are.  It  has  al- 
lowed us  to  create  an  instructional  organi- 
zation that  we  didn't  have  before  and  to 
learn  how  to  put  things  into  a portfolio. 
That's  an  important  step  for  us. 


ET 


Interviewed  by  Melody  Schneider. 

Reprinted  with  permission  from  Phenomenal  Changes,  Literacy  South,  Durham,  NC. 
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Students  Connecting  with  Students 

Lessons  in  Health  Care 


’ W A Thoarewe? 

1/m/  We  are  a group  of  eight 

W * women  from  different  places 
with  different  languages  and  backgrounds: 
Dominican,  Haitian,  Mexican,  Vietnamese, 
Russian,  and  American.  We  get  together  to 
work  on  health  issues  such  as  breast  can- 
cer, HTV  and  AIDS,  and  violence  because 
we  want  our  community  to  be  safe  and  to 
be  healthy.  We  are  called  The  Student  Ac- 
tion Health  Team  and  got  started  in  1994, 
funded  by  the  Comprehensive  Health 
Project  from  the  Massachusetts  Depart- 
ment of  Education  to  do  health  education 
at  Operation  Bootstrap,  where  we  are  or 
have  been  students. 

Operation  Bootstrap  is  located  in  Lynn, 
MA  and  offers  classes  in  Adult  Basic  Edu- 
cation (ABE),  English  for  Speakers  of  Other 
Languages  (ESOL),  and  General  Educa- 
tional Development  (GED)  as  well  as  Fam- 
ily Literacy  and  pre-vocational  programs. 
There  are  about  200  students  in  all  of  Boot- 
strap's programs  and  classes. 

As  The  Student  Action  Health  Team 
our  job  is  to  find  out  about  the  health 
education  needs  of  students  at  Operation 
Bootstrap  and  develop  plans  to  meet  some 
of  these  needs.  Every  year  we  recruit  new 
members  for  the  team.  Right  now  there  are 
four  original  members  and  four  new  mem- 
bers. 

After  our  first  year  together,  we  de- 
cided that  we  needed  to  do  direct  teaching 
in  both  English  and  native  languages 
about  health  in  the  Bootstrap  ABE,  ESOL> 
GED,  and  Family  Literacy  classrooms  be- ' 
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cause  most  community  health  education 
efforts  did  not  meet  the  needs  of  Bootstrap 
students.  We  had  brought  in  some  commu- 
nity health  educators  during  the  first  year 
but  they  talked  too  fast  with  difficult  lan- 
guage and  the  materials  they  brought  were 
too  difficult  to  understand.  Bootstrap  stu- 
dents did  not  feel  safe  to  ask  questions  or 
talk  about  whether  the  information  was 
useful  in  their  lives.  The  students  told  us 
they  felt  empty  inside. 

We  decided  we  should  let  students 
choose  the  health  topics  they  want  to  know 
more  about.  The  team  does  this  by  putting 
on  a health  fair  every  fall  for  Bootstrap  stu- 
dents and  staff.  We  put  up  a list  of  health 
issues  and  then  each  student  and  staff 
member  places  one  dot  next  to  the  issues 
they  are  most  interested  in  learning  about. 
In  1995-96,  the  top  vote  was  for  cancer 
education  and  we  decided  to  focus  on 
breast,  cervical  and  testicular  cancer  be- 
cause if  they  are  detected  early,  successful 
treatment  is  more  likely.  In  1996-97,  the 
top  vote  was  for  violence  prevention.  Boot-, 
strap  students  were  terrified  because  of  the 
abduction  of  6-year-old  Jesus  De  La  Cruz 
and  this  made  them  want  to  learn  more 
about  violence  prevention  in  the  commu- 
nity and  in  the  home. 

The  educational  programs  are  carried 
out  in  the  classrooms  as  part  of  the  regular 
classes.  Our  programs  have  three  sessions 
of  about  1.5  hours  each  and  provide  basic 
facts,  hands-on  practice,  and  information/ 
discussion  about  community  resources. 

We  use  lots  of  drama  to  get  at  difficult  and 


by  Judy  Berry, 
Vandal  Ivaneko, 
Nguyet  Nyugen, 
Ana  Reynoso, 
Elsa  Reynoso, 
Magali  Torres  and 
Charlene  Wigfall. 
Marcia  Hohn  is  the 
team  facilitator. 


Operation  Bootstrap 
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sensitive  issues.  Drama  has  been  a good 
way  to  break  the  ice  between  the  team  and 
Bootstrap  students  and  to  get  to  students' 
hearts.  We  also  use  methods  such  as 
agree/ disagree  exercises,  small  group 
discussions,  and  pictures  to  get  students 
and  teachers  participating,  discussing, 
and  interacting  with  the  information. 

For  example,  in  the  cancer  education 
program,  we  held  up  written  statements 
about  basic  facts  and  asked  students  to 
"vote"  whether  they  thought  the  statement 
was  true  or  false  and  then  discussed  the 
"answer"  as  it  is  known  today.  In  begin- 
ning English  classes  we  translated  this  in- 
formation into  as  many  as  six  languages. 
We  wanted  to  make  sure  everyone  really 
understood  the  information  and  had  a 
chance  to  tell  their  stories  and  a safe  atmo- 
sphere in  which  to  ask  questions. 

In  the  next  session  small  groups  fea- 
tured hands-on  practice  with  breast  and 
testicular  models.  A cancer  educator 
helped  us  and  we  asked  her  to  work  with 
the  men  because  we  felt  shy  about  teaching 
the  men  how  to  examine  their  testes. 

In  beginning  English  classes,  we  worked 
in  small  groups  by  language.  The  last  ses- 
sion was  a drama  about  going  to  the  doctor 
and  showed  a situation  where  a patient 
with  no  insurance  and  limited  English  is 
not  treated  respectfully  by  the  health  care 
providers.  This  opened  up  the  discussion 
about  students'  fears  about  discrimination 
and  we  talked  about  rights  and  responsi- 
bilities in  medical  situations. 

We  used  drama  a lot  in  the  Family 
Violence  program  to  illustrate  the  stories 
strangers  use  to  get  innocent  children  to 
come  with  them  and  to  illustrate  the  short- 
and  long-term  effects  of  witnessing  vio- 
lence in  the  home  on  children.  Our  dramas 
— called  Tommy  at  Eight  Years  and 
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Tommy  at  14  Years  — were  about  a boy 
from  a violent  home  and  illustrated  the 
long-term  effects  of  witnessing  violence. 
Students  discussed  the  drama  afterward 
and  gave  their  ideas  about  what  could  help 
Tommy.  We  also  performed  a man-woman 
violence  drama  with  a similar  small  group 
discussion.  The  idea  is  to  get  students  talk- 
ing about  the  health  issue,  to  help  each 
other  and  to  find  community  resources 
that  help.  All  this  must  take  place  in  a safe 
atmosphere  where  students'  privacy  and 
cultural  beliefs  are  respected. 

Every  year  we  assess  our  program  to 
find  out  what  the  Bootstrap  students 
thought  about  it  and  what  changes  in 
knowledge,  attitudes  and  actions  hap- 
pened. As  program  coordinators,  we  also 
reflect  about  what  we  learned,  how  we 
changed  and  how  we  saw  the  program 
affect  other  people.  We  want  to  tell  you 
how  we  do  this. 

ASSESSING  THE  PROGRAM  IN  EARLY 
DETECTION  OF  BREAST,  CERVICAL  AND 
TESTICULAR  CANCER  1995-1996 

For  our  assessment,  we  decided  to  conduct 
a student  survey  to  see  the  results  of  the 
work  the  team  had  done  in  early  detection 
of  breast,  cervical  and  testicular  cancer. 

We  developed  the  survey  by  brain- 
storming all  we  wanted  to  know,  writing 
these  thoughts  down  on  Post-Its.  For  ex- 
ample, we  wanted  to  know  if  students  felt 
they  understood  the  information,  if  they 
liked  our  teaching  methods,  if  they  saw 
this  information  as  important  to  them  and 
their  families.  We  grouped  the  Post-Its  to 
create  categories.  We  then  worked  in  pairs 
to  develop  statements  that  Bootstrap  stu- 
dents could  agree  or  disagree  with  on  a 
scale  of  1-5.  There  were  three  categories: 

1)  the  importance  of  learning  about  breast, 
cervical  and  testicular  cancer  at  Bootstrap, 
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2)  the  effectiveness  of  the  teaching,  and  3) 
resulting  actions  or  steps  the  students  had 
taken  (such  as  having  gone  for  a PAP  test 
or  a mammogram,  or  doing  self-exams). 
Each  category  had  six  statements  or  ques- 
tions. 

We  distributed  the  survey  to  the  stu- 
dents in  their  classrooms,  then  we  com- 
piled the  results  and  put  it  in  bar  graph 
form,  which  we  thought  the  students  could 
easily  understand.  The  teachers  helped  us 
by  explaining  how  to  fill  out  the  survey 
since  many  students  were  not  familiar 
with  the  use  of  1-5  scales.  Although  many 
students  who  had  participated  in  the  pro- 
gram had  already  left,  42  students  filled 
out  the  survey.  Here  is  an  example  of  one 
of  the  bar  graphs. 

ACTIONS  TAKEN  BY  LITERACY  STUDENTS 
AT  OPERATION  BOOTSTRAP  TWO  MONTHS 
AFTER  THE  BREAST,  CERVICAL,  AND 
TESTICULAR  CANCER  EDUCATION  PROGRAM 
BY  THE  STUDENT  ACTION  HEALTH  TEAM 
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The  survey  showed  that  the  students  felt 
they  had  learned  health  information  im- 
portant for  their  lives  and  the  lives  of  their 
families  and  friends,  they  liked  having 
other  students  as  their  teachers  and  appre- 
ciated the  many  different  teaching  tech- 
niques, especially  the  drama.  The  survey 
further  showed  that  students  were  taking 
action  as  a result  of  the  education  program. 

We  also  conducted  an  internal  team 
evaluation  about  how  we  felt  about  the 
work  we  did.  We  drew  pictures  and  made 
statements  about  our  personal  feelings  and 
shared  these  things  in  the  team.  Eighteen 
positive  statements  were  reported,  such  as 
school  being  the  best  place  to  learn  about 
health  and  that  the  information  went  be- 
yond the  classroom  to  family  and  friends. 
We  were  proud  of  each  other  and  saw  that 
we  were  role  models  for  each  other  as  well 
as  for  the  Bootstrap  students.  We  appreci- 
ated the  support  of  Marie  Wallace  (who  is 
a cancer  educator).  She  informed  us  about 
cancer. 

We  also  appreciated  the  support  of  the 
teachers  working  alongside  the  team  by 
preparing  students  with  vocabulary  and 
following  up  with  additional  activities. 
Overall,  we  saw  that  receiving  information 
in  a native  language  and/or  in  simple 
ways  with  simple  terms  made  for  better 
understanding.  It  was  our  opinion  that  be- 
cause students  had  better  understanding 
they  were  able  to  take  action  such  as  doing 
self-exams  or  going  for  free  PAP  tests  and 
mammograms. 

ASSESSING  THE  PROGRAM  ON 
FAMILY  VIOLENCE  1996-97 

We  decided  to  do  one-to-one  interviews 
to  find  out  about  the  effects  of  the  family 
violence  program  because  this  is  a per- 
sonal and  very  sensitive  issue.  Many  Boot- 
strap students  have  this  problem  in  their 
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lives.  Two  Operation  Bootstrap  women 
students  have  been  killed  by  their  partners 
over  the  past  few  years.  Many  others  live 
in  fear  and  in  threatening  situations.  Also, 
Bootstrap  students  were  very  emotionally 
affected  by  the  abduction  of  Jesus  De  La 
Cruz  in  September  of  1996.  (He  remains 
missing  today.) 

We  went  to  the  students  one  by  one  af- 
ter the  second  session  of  our  program  - the 
effects  on  children  of  witnessing  violence 
in  the  home  - and  did  36  interviews.  Team 
members  went  to  the  classrooms  and  ex- 
plained that  we  were  evaluating  our  pro- 
gram and  needed  students'  opinions. 

Many  students  volunteered  to  be  inter- 
viewed. They  told  us  they  learned  that 
children  are  emotionally  affected  for  life, 
continue  the  pattern  of  violence,  and  are 
more  likely  to  use  drugs,  be  runaways,  or 
have  an  early  pregnancy  - and  that  they 
did  not  know  these  facts  before.  For  young 
students  this  was  very  important  informa- 
tion for  their  future  lives.  After  the  man- 
woman  violence  drama  we  conducted 
some  additional  interviews  and  we  were 
very  careful  to  let  students  know  we  were 
not  singling  anyone  out,  that  we  needed  to 
get  their  thoughts  so  we  could  improve 
our  teaching. 

Students  told  us  they  liked  the  use  of 
drama,  that  it  is  a good  way  to  present  in- 
formation. Family  violence  is  a hard  topic 
to  talk  about  and  drama  keeps  it  safe  so 
people  do  not  feel  singled  out.  The  stu- 
dents liked  the  opportunity  to  give  opin- 
ions in  small  group  discussions  after  the 
dramas  about  how  to  help  stop  the  cycle  of 


violence. 

We  also  learned  that  students  were  tak- 
ing action.  They  were  sharing  the  informa- 
tion with  family,  neighbors,  friends  and 
co-workers,  learning  how  to  use  neighbors 
and  the  phone  to  call  the  police  for  protec- 
tion, and  using  community  agencies  for 
assistance  when  necessary.  Students  were 
also  using  religion,  support  groups,  and 
community  counselors.  Many  students 
told  vis  that  they  did  not  know  about  these 
resources  before,  especially  resources  for 
helping  with  children.  Students  within  the 
program  were  also  trying  to  help  each 
other. 

In  our  assessments  of  both  the  cancer 
and  violence  education  programs,  students 
told  us  that  Operation  Bootstrap  is  a good 
place  to  learn  about  health  issues  and  that 
health  education  added  to  their  language 
and  literacy  learning.  Teachers  told  us  that 
students  got  so  interested  in  talking  about 
health  issues  in  the  class  that  they  would 
forget  to  worry  about  their  English  — they 
would  just  talk  in  English  even  if  it  was 
not  perfect  because  they  were  so  anxious 
to  communicate  about  the  topic. 

Several  students  told  us  that  the  Student 
Action  Health  Team  should  teach  in  other 
places  because  we  "know  how  to  teach 
and  teach  important  things."  This  made  us 
feel  very  proud.  During  1997-98  we  will  be 
mentors  to  programs  new  to  doing  health 
education  in  their  program.  We  have  a lot 
of  experience  and  lessons  to  share  and  we 
look  forward  to  helping  them  with  the  im- 
portant topic  of  health. 
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National  Center  for  the  Study  of  Adult  Learning  and  Literacy  (NCSALL) 

Assessment  Research  Agenda 


What  impact  does  participation 
in  adult  learning  and  literacy 
programs  have  on  an  adult's 
life  and  how  can  this  impact  effectively  be 
assessed? 

The  NCSALL  partners  at  the  Center  for 
Literacy  Studies  (CLS)  at  the  University  of 
Tennessee  and  the  College  of  Education  at 
Rutgers  University  began  working  with 
this  question.  In  the  first  year  of  NCSALL 
(the  federally-funded  National  Center  for 


the  Study  of  Adult  Learning  and  Literacy) 
we  have  been  getting  our  research  projects 
up  and  running,  and  beginning  to  get  a 
handle  on  this  question.  It  is  more  complex 
than  we  ever  imagined! 

We  are  currently  working  on  four 
projects  that  we  hope  will  help  us  move  to- 
ward an  answer.  The  first  is  a policy  study 
that  examines  existing  impact  studies  and 
state  performance  accountability  projects 
and  systems  and  draws  implications  for 
literacy  policy  and  practice  and  for  the 
design  and  methodology  of  future  impact 
studies.  The  two  papers  in  this  study  are 
being  written  by  Hal  Beder  and  Juliet 
Merrifield  and  should  be  available  around 
the  first  of  the  year.  The  papers  make  clear 
that  answering  questions  like,  "What  dif- 
ference does  our  work  make?"  and  "Is  it 
working?"  depends  on  who  is  asking  and 
for  what  reasons  as  well  as  who  is  answer- 
ing based  on  what  data  and  what  assump- 
tions. 

The  second  study  is  looking  at  how 
adult  learners  identify  the  impacts  of  par- 
ticipation in  adult  education  in  their  lives. 
We  are  currently  interviewing  ten  Tennes- 
see adults  who  have  been  students  in  adult 


basic  education.  We  are  asking  them  to  tell 
us  about  their  lives  — about  their  work, 
families,  and  community  activities  as  well 
as  their  educational  experiences.  We  plan 
to  follow  these  initial  interviews  with 
interviews  with  another  fifty  adult  learners 
from  a variety  of  programs  across  the 
country.  From  these  life  stories  we  hope 
to  be  able  to  "hear"  the  differences,  the 
impacts  that  adult  education  have  had  in 

fVioit*  1 itrno 

A third  lens  for  looking  at  the  larger 
question  is  that  of  local  programs.  We  will 
be  working  in  an  action  research  project 
with  a Tennessee  program  that  is  imple- 
menting the  new  Equipped  for  the  Future 
framework  as  it  develops  new  ways  of 
monitoring  the  program.  We  have  re- 
viewed how  other  fields  (public  health, 
community  development)  measure  out- 
comes, particularly  improvement  in  qual- 
ity of  life  of  individuals  and  communities. 
We  will  be  working  with  this  literacy  pro- 
gram to  discover  what  can  be  applied  to 
measuring  changes  in  the  lives  of  learners. 

The  NCSALL  partners  at  Rutgers  are  be- 
ginning to  identify  the  variety  of  teacher/ 
learner  transactions  or  interactions  that 
occur  in  adult  education  classrooms.  With 
this  understanding  we  will  be  better  able 
to  know  what  it  is  that  adult  learners  have 
participated  in. 

When  we're  done,  what  will  we  have? 
We  know  for  sure  that  we  will  not  have 
the  answer  to  our  big  question.  But  we  will 
have  more  answers.  And  more  questions. 
We  will  have  looked  closely  at  what  has 
and  hasn't  worked  in  previous  attempts  to 
answer  these  questions.  We  will  have 
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heard  the  voices  of  learners  talking  about 
impacts  on  their  lives.  We  will  have  a 
model  of  looking  at  outcomes  of  adult  edu- 
cation in  the  quality  of  learners'  lives,  a 
model  that  can  be  used  by  local  programs. 
We  will  have  a better  understanding  of 
what  is  happening  in  adult  education 
classrooms  in  order  to  better  approach 
measuring  the  outcomes. 

While  I know  from  experience  the  many 
challenges  of  recruitment,  teaching,  and 
staff  development,  I never  had  any  diffi- 
culty thinking  about  these  areas  of  practice. 

But  thinking  about  assessment  is  a dif- 
ferent matter.  We  began  by  trying  to  get 
clear  on  the  language:  inputs,  outputs,  out- 
comes, impact,  indicators  of  program  qual- 
ity, performance  accountability  measures. 
We  tried  to  find  out  what  these  mean  and 
ended  up  deciding  that  the  best  we  could 
do  was  to  decide  what  we  mean  by  them. 

The  questions  of  what  to  assess  and  for 


what  reasons  need  as  much  thinking  as 
how  to  assess.  Do  we  want  to  know  what 
happened,  or  do  we  want  to  know  did  x,  y, 
or  z happen?  We  all  want  to  know  the  im- 
pact of  our  work  — as  teachers,  as  learn- 
ers, as  program  administrators,  as  funders. 
But  each  of  us  have  different  ways  of  ask- 
ing the  question.  "Did  they  learn  what  I 
taught?"  "What  did  I learn?"  "How  many 
passed  the  G.E.D.?"  "How  many  got  a 
job?"  "What  difference  did  any  of  this 
make  in  my  life?"  "What  difference  does 
any  of  this  make  in  my  community?"  Un- 
derlying all  these  questions  is  the  question 
of  who  gets  to  decide. 

So  one  year  into  our  work  on  impacts 
assessment,  we  are  back  to  doing  the  work 
on  our  projects.  We  hope  the  research  will 
contribute  to  a much  bigger  discussion 
about  assessment  that  will  help  us  clarify 
what  we  can  say  about  our  impact  as  a 
field. 
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Voices  from  the  Field:  The  Basic  English  Skills  Test 


In  this  section  we  look  at  various  voices  from  the  field  around  the  much-used  BEST . Although  Adventures  is  a 
journal  about  learner-centered  approaches  to  assessment  and  evaluation,  it  is  time  to  look  more  deeply  at  other 
"assessment  tools"  that  we  all  use . This  section  is  not  intended  to  encourage  or  discourage  the  use  of  the  BEST,  but 
to  ask  practitioners  in  the  field  how  they  view  the  test  from  the  historical,  teacher,  and  volunteer  perspectives . We 
should  all  reflect  on  the  tools  we  use  to  see  if  they  fit  with  our  programs  and  our  learners. 


The  History  of  the  BEST 


The  history  of  the  BEST  is  an  inter- 
esting one.  It  began  in  the  early 
1980s  when  the  Federal  Office  of 
Refugee  Resettlement  (ORR)  saw  a need  to 
more  effectively  move  refugees  from  wel- 
fare to  economic  self  sufficiency  through 
the  "provision  of  a coordinated  and  struc- 
tured English  Language  Training  Pro- 
gram." In  order  to  make  this  happen,  they 
invested  in  a national  initiative  called  the 
Mainstream  English  Language  Training 
(MELT)  Project  which  brought  together 
ESL  professionals  from  throughout  the  US. 
The  goal  of  this  initiative  was  to  develop 
and  design  not  only  a test,  but  curriculum 
guidelines  and  Student  Performance  Level 
(SPL)  definitions. 

The  International  Institute  of  Boston 
was  one  of  the  contributors  to  the  MELT 
Project.  As  part  of  the  Project,  the  BEST 
(Basic  English  Skills  Test)  was  developed 
and  field  tested  in  a number  of  locations, 
including  Boston.  This  test  was  designed 
to  assess  "elementary"  listening,  speaking, 
reading  and  writing  skills  reflecting  a com- 
petency based  approach  to  language  teach- 
ing and  assessment.  The  test  could  be 
reliably  used  if  an  individual  had  low  lit- 
eracy skills.  The  oral  test  could  be  used  by 
itself  if  an  individual  did  not  have  the  lit- 
eracy skills  needed  for  the  written  compo- 
nent. 

The  BEST  is  life  skills-  and  task-based, 
evaluating  a student's  ability  to  use  En- 
glish in  real  life  situations.  It  was  thought 
to  be  a tool  that  could  provide  useful  infor- 
mation in  determining  an  initial  class 
O 
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placement  or  determining  progress  in 
some  specific  functional  areas  such  as  tell- 
ing time  or  counting  money.  It  was  never 
intended  to  test  general  language  profi- 
ciency or  to  be  especially  useful  in  pro- 
grams that  did  not  use  a survival  life 
skills-based  curriculum. 

Much  of  the  ESL  being  offered  in  the 
early  1980s  was  not  life  skills-based  and 
did  little  to  introduce  learners  to  the  world 
of  work.  Often  the  emphasis  was  grammar 
rather  than  functional  language  skills. 

Even  more  important  to  teachers,  there 
were  no  resources  for  working  with  adults 
who  were  non-literate  or  had  low  native 
language  literacy  skills. 

Likewise  ESL  assessment  tools  tended 
to  be  more  academic  with  an  emphasis  on 
testing  grammar.  Very  few  tests  could  ef- 
fectively assess  learners  with  low  literacy 
and/or  little  or  no  knowledge  of  English. 
As  a result,  many  of  the  materials  being 
used  in  the  field  were  either  inappropriate 
or  were  being  generated  by  teachers  with 
few  guidelines  or  standards. 

ORR,  together  with  ESL  professionals 
from  many  parts  of  the  United  States, 
decided  there  was  a need  to  develop  some 
common  tools  and  common  language  for 
the  Refugee  ESL  programs  which  were 
overwhelmed  by  the  large  numbers  of 
adults  arriving  each  month  needing  both 
English  and  the  skills  to  get  jobs. 

Another  interesting  fact  about  the  MELT 
Project  is  its  relationship  to  the  oversees 
refugee  camp  programs.  In  the  late  1970s 
and  early  1980s  following  the  Cambodian 
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genocide  and  the  aftermath  of  the  Vietnam 
war,  large  number  of  refugees  were  filling 
camps  in  countries  throughout  Southeast 
Asia.  Many  of  them  were  applying  to  emi- 
grate to  the  United  States.  The  Department 
of  State  funded  processing  centers  in  a 
number  of  sites  in  both  Southeast  Asia  and 
Africa  where  ESL  and  orientation  prior  to 
arrival  to  the  U.S.  was  required  for  all  adults 
accepted  into  the  U.S.  refugee  program. 

Close  to  35,000  people  graduated  from 
these  programs  each  year.  In  places  like 
Galang,  Indonesia,  Bataan,  Philippines  and 
Phanat  Nikon,  Thailand,  programs  were 
set  up  and  intensive  efforts  were  made  to 
develop  common  curriculum,  common 
assessment  processes  and  uniform  class 
level  definitions.  As  a result,  the  curricu- 
lum development,  staff  training  and 
resources  produced  in  these  overseas  pro- 
grams was  impressive. 

Even  more  impressive  was  that  the 
work  done  in  these  refugee  camps  was  co- 
ordinated with  the  MELT  material  devel- 
oped stateside.  For  example,  when  a 
refugee  arrived  in  the  U.S.,  his  or  her  class 
level  was  stamped  on  the  1-94  card  giving 
stateside  ESL  programs  an  indication  of  a 
learner's  SPL  upon  completion  of  the  over- 
seas program.  Since  the  BEST  was  corre- 
lated with  the  SPL,  it  was  possible  to  verify 
an  SPL  using  the  BEST  and  more  easily 
place  a student  into  a class.  Overseas  ESL 
and  Cultural  Orientation  curriculum  was 
shared  with  teachers  in  the  U.S  so  teachers 
often  knew  what  material  students  had 
covered  in  the  camps;  the  MELT  curricu- 
lum was  designed  to  build  on  what  was 
covered  in  the  camps. 

Now,  almost  12  years  later,  the  BEST 
and  other  MELT  material  have  gone  well 
beyond  ESL  programs  targeting  incoming 
refugees.  As  accountability  and  outcomes 
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measurement  become  increasingly  impor- 
tant, many  funders  have  adopted  the  SPL 
and  the  BEST  as  required  tools  for  ESL 
programs.  Obviously  the  framework  and 
"common  language"  that  these  tools  pro- 
vide give  funders  a better  ability  to  quan- 
tify progress. 

At  a program  level,  however,  assess- 
ment has  to  carefully  consider  both  learner 
and  program  goals.  So  while  the  BEST  is 
often  thought  to  be  one  of  the  better  tests, 
especially  for  learners  at  the  beginning  lev- 
els, it  is  not  adequate  for  all  programs  and 
learners.  The  test  was  developed  as  an 
alternative  to  traditional  paper  and  pencil 
tests,  with  an  emphasis  on  assessing  very 
basic  life  skills.  If  a program  is  not  working 
with  a basic  survival  life  skills  focus,  this 
test  may  not  be  particularly  useful  when 
determining  initial  class  placement  or  when 
measuring  progress.  If  the  emphasis  is  pre- 
vocational,  for  example,  more  specific  pre- 
vocational  skills  may  need  to  be  measured 
rather  than  survival  skills.  In  addition,  for 
learners  who  have  lived  in  the  U.S  for  a 
long  time,  this  test  may  not  be  appropriate. 
Tasks  like  telling  time,  counting  money, 
writing  checks  or  circling  dates  on  a calen- 
dar may  simply  not  be  challenging  enough. 

Programs  often  use  the  BEST  in  conjunc- 
tion with  other  tools.  This  enables  teachers 
to  individualize  testing  by  drawing  on  a 
variety  of  tools,  not  just  one  test.  Testing 
needs  to  match  both  the  learner  and  where 
he/ she  is  coming  from  culturally,  linguisti- 
cally, and  academically  and  what  the  pro- 
gram needs  to  know  to  adequately  address 
the  learning  needs  of  the  student.  The  BEST 
by  itself  may  or  may  not  satisfy  all  of  this. 

Taken  from  Competency-Based  Mainstream 
English  Language  Training  Project  (MELT), 

U.S.  Department  of  Health  and  Human  Services, 
Social  Security  Administration,  Office  of  Refugee 
Resettlement  (March  1985),  p.  1,  section  1. 
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Why  I Think  the  BEST 
Isn't  Good  Enough 


According  to  its  documentation,  the 
BEST  was  designed  for  determin- 
ing classroom  placement,  for  as- 
sessing individual  or  class  progress,  and 
diagnosing  "language-use  tasks"  (like  tell- 
ing time)  that  need  teaching  and  remedia- 
tion. 

As  a placement  tool,  the  BEST  attempts 
to  cover  a lot  of  ground,  including  not  only 
grammar  and  vocabulary  but  also  cultural 
familiarity  and  the  American  idiom.  Were 
all  prospective  students  nearly  alike  in 
background,  the  BEST  might  be  more  use- 
ful. But  consider  two  possibilities:  a Rus- 
sian immigrant  with  post-secondary 
education(including  formal  English  stud- 
ies) who  arrived  in  the  US  two  days  ago; 
and  an  immigrant  from  the  Dominican  Re- 
public who  completed  only  primary  school 
and  has  lived  in  the  US  for  twelve  years, 
all  the  while  working  on  an  assembly  line 
with  little  opportunity  for  contact  with 
native  speakers. 

The  Russian  is  likely  to  miss  all  the 
money  questions,  never  having  seen 
American  currency  before  the  test;  most 
of  the  other  questions  will  be  incompre- 
hensible because  the  examiner's  American 
accent  is  so  unlike  the  British  accent  the 
Russian  encountered  in  previous  studies. 
Yet,  this  person  may  have  considerable 
vocabulary  and  exposure  to  written 
English,  not  to  mention  the  benefit  of 
having  already  studied  English. 

Meanwhile,  the  factory  worker  has  been 
shopping,  watching  television,  and  seeing 
her  children  through  school.  She  has 
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learned  to  speak  enough  pidgin  English  to 
get  through  day-to-day  language  encoun- 
ters, as  well  as  several  questions  on  this 
test. 

Consider  the  possible  core  evaluation 
scores  for  these  two  students.  The  newly- 
arrived  Russian  is  so  mystified  by  the 
American  accent  that  she  can  understand 
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names  objects  in  each  picture  of  the  test. 
She  scores  eight  points,  although  with  a 
couple  of  trips  to  the  grocery  store  she 
might  have  gotten  another  four  or  five 
points  for  the  shopping  portion  of  the  test. 
Her  core  evaluation  results:  SPL  0 (Begin- 
ning literacy  ESOL). 

The  second  student's  performance 
reflects  cultural  familiarity  and  a lot  of 
(fossilized  and  incorrect)  language.  She 
gets  five  points  for  the  shopping  portion  of 
the  test,  she  does  well  on  the  fluency  ques- 
tions (where  volume  of  production  is  mea- 
sured without  penalty  for  grammatical 
inaccuracy),  earning  13  more  points.  These 
18  points  place  her  at  SPL  2 (Beginning 
ESOL).  Both  students  are  in  some  sense 
beginners,  but  their  language  needs  are 
very  different.  The  difference  is  far  more 
complex  than  10  points  worth. 

As  a student  gains  more  language,  the 
scoring  of  the  BEST  test  becomes  more 
punishing.  The  listening  vocabulary 
needed  to  earn  the  nine  "listening  compre- 
hension" points  is  not  difficult,  although 
an  inability  to  follow  a map  may  cost  even 
an  advanced  student  two  of  these  points. 
Eight  questions  are  scored  for  fluency.  The 
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three  points  for  a " fluency  question"  are 
accessible  to  a student  who  has  only  a 
minimal  degree  of  grammatical  control 
but  who  is  expressive  and  uninhibited  by  a 
concern  for  perfection;  a reticent  personal- 
ity can  easily  diminish  the  score  of  a more 
capable  student.  But  the  24  questions  on 
Form  B that  are  scored  for  grammatical 
and  idiomatic  accuracy  are  truly  problem- 
atic for  the  ESOL  student. 

Consider  the  question  "What  is  she 
doing?"  accompanied  by  a picture  of  a 
woman  watching  TV.  "She  watch  TV," 
"Her  watching  TV,"  and  "She  watching 
TV"  all  get  scored  as  a one-point  answer 
(correct  information  in  a response  that 
would  not  be  used  by  a native  speaker), 
while  "Watching  TV"  is  a two-point  answer. 
For  these  "communication  questions" 
there  is  a one-point  risk  in  attempting  any- 
thing more  sophisticated  than  the  phrasal 
response  "watching  TV."  A two-point 
answer  requires  subject-verb  agreement, 
verb  tense,  articles,  prepositions  and  word 
choice  being  grammatically  and  idiomati- 
cally correct,  too.  For  a student  who  is  in  a 
class  that  encourages  communication  over 
grammatical  precision,  the  BEST  changes 
the  rules  by  awarding  more  points  when 


the  student  gives  an  answer  terse  enough 
that  the  inevitable  slip  of  article  or  preposi- 
tion never  occurs.  The  BEST  fails  to  cap- 
ture how  well  the  student  has  mastered 
any  of  the  pitfalls  that  lie  between  "watch 
TV"  and  "she's  watching  TV." 

The  BEST'S  third  goal  — as  a diagnostic 
tool  — is  only  somewhat  better  met.  The 
BEST  can  point  out  which  of  the  several  ar- 
eas of  "survival"  , vocabulary  (and  culture) 
are  needed  by  a student.  However,  given 
its  all-or-nothing  approach  to  grammatical 
accuracy,  it  is  of  no  use  in  determining  the 
structural  issues  of  English  that  need  to  be 
addressed; 

The  MELT  curricula  framework,  of 
which  the  BEST  was  the  final  piece,  defined 
the  task  of  the  ESOL  class  as  being  tied  to 
"language  use  tasks"  in  the  American  cul- 
ture, necessary  for  functioning  in  the  US. 

It  is  a useful  tool  for  the  ESOL  teacher.  The 
long  form  of  the  BEST  is  a cumbersome  at- 
tempt to  quantify  language  skills  that  fails 
to  account  for  the  diversity  of  our  students' 
backgrounds  and  for  the  immensity  of  the 
language  learning  task.  An  ESOL  program 
would  do  well  to  consider  other  tools  for 
placement,  progress,  and  diagnosis  of  the 
language  learners'  needs. 
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The  BEST  is  Workable, 

But  It's  Not  the  Only  Choice 


I began  using  the  BEST  in  the  early 
1980s  when  I was  an  ESL  coordinator 
for  the  Refugee  Employment  and  Edu- 
cation Program  (REEP)  in  Massachusetts. 

I used  it  to  help  assess  the  oral  English  lan- 
guage skills  of  the  newly-arriving  adult 
refugees. 

At  that  time,  there  was  only  one  version 
each  of  the  core  section  and  die  literacy 
skills  section.  The  program  was  a compe- 
tency-based program  which  taught  sur- 
vival skills  and  pre-vocational  skills,  and 
the  BEST  tested  these  areas.  It  provided  a 
baseline  score  which  could  be  compared  to 
all  learners  for  placement  purposes.  When 
used  as  an  on-going  assessment  tool,  it 
tested  mastery  of  the  materials,  showing 
the  need  for  more  practice,  movement  to 
a higher  level  or,  ultimately,  ability  to 
handle  employment. 

When  used  in  this  manner,  the  BEST 
was  quite  effective,  but  its  biggest  draw- 
back was  that  it  took  too  long  to  adminis- 
ter. The  core  section  had  to  be  adminis- 
tered to  each  student  individually,  which 
often  took  half  an  hour.  Only  several  years 
later  did  a short  version  appear  which 
could  be  given  in  ten  to  fifteen  minutes. 

The  literacy  skills  section  was  still  rarely 
used  because  it  took  at  least  an  hour  and 
the  program  emphasized  aural/oral  skills  . 

In  the  late  1980s,  refugee  programs  were 
funded  through  local  Service  Delivery  Ar- 
eas (SDAs)  which  had  a central  assessment 
unit  that  administered  the  BEST  to  all  pro- 
gram participants.  Individuals  who  never 
used  the  BEST  or  had  no  experience  testing 
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limited  English-proficient  clients  were  now 
responsible  for  program  referral  and  place- 
ment. I was  asked  by  the  local  SDA  to  train 
their  assessment  staff  in  the  use  of  the 
BEST.  However,  since  this  subjective  test 
was  open  to  the  opinion  of  the  tester,  the 
scores  often  made  no  sense.  Students  who 
could  repeat  their  personal  information, 
tell  time,  and  identify  money  would  score 
higher  than  they  really  were  in  overall  En- 
glish Language  Skills.  Conversely,  stu- 
dents soon  learned  that  high  scores  could 
prevent  them  from  attending  ESL  classes, 
so  they  often  would  not  answer  questions 
in  order  to  score  low.  As  a result,  I had  to 
develop  an  in-house  assessment  tool  which 
would  be  more  reliable  for  placement  pur- 
poses. 

Most  recently  I was  still  associated  with 
a refugee  education  program,  and  assess- 
ment has  gone  back  to  the  programs.  The 
BEST,  in  its  long  form,  is  still  the  chosen 
tool  of  assessment.  It  is  used  for  placement 
purposes,  but  it  is  also  used  as  the  only  on- 
going assessment  tool  to  identify  student 
achievement.  The  same  version  of  the  test 
was  administered  every  ten  weeks  (until 
recently  the  test  was  given  every  five 
weeks). 

Since  the  success  of  the  program  is 
linked  to  the  increase  in  the  number  of  Stu- 
dent Performance  Levels  (SPLs) , the  cur- 
riculum is  centered  around  the  basic  skills 
tested  in  the  test.  When  a student  reaches 
an  SPL  level  6,  he  is  no  longer  eligible  for 
English  classes.  This  has  led  to  many  ques- 
tions about  the  validity  of  the  test  in  my 
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mind.  While  participating  in  the  program, 
a student  could  take  the  same  test  six  or 
more  times.  Also,  many  people  overlook 
the  fact  that  this  is  an  oral  interview  for  as- 
sessing basic  life  skills.  If  you  have  been  in 
this  country  for  a year  or  more,  you  most 
likely  will  be  able  to  answer  questions 
about  survival  topics  but  do  you  really 
have  enough  language  skills  to  success- 
fully obtain  employment  or  advance  in 
your  position? 

I do  understand  and  support  an  "early 
to  work"  philosophy  for  limited  English 
speakers,  but  I feel  that  students  should  be 
allowed  to  continue  their  education  to 
achieve  advancement.  The  BEST  is  a good 
assessment  tool  for  placement,  but  it 
should  not  be  the  only  tool  used  to  show 


individual  achievement  or  program  effec- 
tiveness. In  the  late  1980s  when  the  Federal 
government  passed  legislation  calling  for 
standardized  tests  to  evaluate  ABE  and 
ESL  programs,  there  was  much  debate 
about  the  purpose  of  assessment  and  inter- 
est in  the  field  for  alternative  assessment 
tools.  The  conclusions  made  were  that  as- 
sessment should  focus  on  the  needs  of  the 
students  and  measure  how  we  are  helping 
students  meet  their  goals.  It  is  an  ongoing 
partnership  between  the  program  and  the 
student. 

Assessment  can  take  many  forms,  in- 
cluding for  example,  student  portfolios, 
teacher  observation,  student  feedback,  and 
student  participation  in  discussions,  simu- 
lations, and  demonstrations. 
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An  AmeriCorps  Volunteer's 
First  Impressions  of  the  BEST 


I have  been  an  AmeriCorps"' VIST  A 
volunteer  at  the  International  Lan- 
guage Institute  of  Massachusetts  in 
Northampton,  MA  for  three  months.  One 
of  the  most  exciting  parts  of  my  assign- 
ment is  learning  more  about  the  field  of 
ESL  education. 

I was  recently  trained  by  another  VISTA 
volunteer  at  my  site  to  administer  BESTs  to 
incoming  and  current  students.  The  train- 
ing procedure  seemed  quite  lax  to  me:  my 
training  involved  reading  the  BEST  book- 
let to  become  familiar  with  the  questions, 
observing  several  testings,  scoring  the  ob- 
served tests,  and  then  comparing  my 
scores  with  those  of  the  tester.  I noticed 
that  my  trainer  skipped  or  reworded  some 
questions  that  were  in  the  test  booklet. 
Upon  asking  about  this,  my  trainer  ex-' 
plained  to  me  that  those  questions  were 
"worded  too  hard"  and  that  the  woman 
who  trained  her  "didn't  like"  certain  ques- 
tions and  so  skipped  them  or  worded  them 
differently. 

After  several  observations,  I was 
deemed  ready  to  begin  testing  students 
myself. 

Two  groups  of  people  in  our  school 
take  the  BEST.  Incoming  ESL  students  take 
the  BEST  to  determine  in  which  class  level 
they  belong.  Students  already  in  our  pro- 
gram take  the  test  to  determine  their 
progress  and,  if  progress  has  been  made, 
to  reassign  classes. 

It  seems  more  natural  to  administer  the 
BEST  to  incoming  students.  Students  who 
have  never  met  me  or  other  members  of 


the  staff  don't  think  it  strange  if  I ask  them 
their  name,  where  they  are  from,  or  what 
their  native  language  is.  But  it  sometimes 
strikes  current  students  as  strange  to  be 
asked  questions  we  already  know  the 
answer  to  and  have  on  record.  At  best, 
the  student  realizes  this  is  part  of  the  test, 
and  plays  along.  At  worst,  the  student  does 
not  take  the  tester  or  the  test  seriously. 

Students  at  the  ILI  long  enough  to  be 
retested  are  usually  comfortable  enough 
to  want  to  talk  expansively  to  their  tester. 
While  we  certainly  encourage  our  students 
to  share  their  thoughts  and  feelings  with 
us,  while  taking  the  BEST  students  may 
sometimes,  through  conversation,  give 
an  answer  before  the  question  is  asked. 

This  prevents  the  tester  from  determining 
whether  the  student  does  or  would  under- 
stand the  question. 

The  BEST  does  offer  a truer  assessment 
of  communication  skills  than  a multiple 
choice  test.  Because  questions  are  open 
ended,  students  may  answer  questions  in 
a way  that  better  demonstrates  their  skills. 
However,  students  may  speak  with  flu- 
ency and  skill  about  topics  not  covered 
by  test  questions  and  there  is  no  way  for 
students  to  be  "credited"  for  these  skills 
in  the  BEST. 

The  BEST  seems  to  work  better  as  an  as- 
sessment tool  for  those  at  rudimentary  lev- 
els of  English.  Because  it  doesn't  address 
more  sophisticated  English  speaking  skills, 
the  BEST  offers  no  place  to  go  for  a student 
who  has  mastered  basic  communication 
skills.  The  BEST  score  also,  while  it  has  a 
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few  literacy  questions,  does  not  compre- 
hensively test  reading  and  writing  levels, 
so  those  who  have  adequate  speaking 
skills  but  lower  literacy  levels  may  fall 
through  the  cracks. 

The  scoring  of  the  BEST  depends  greatly 
on  the  tester.  While  being  trained,  there 
were  several  instances  in  which  my  trainer 
and  I gave  different  scores  on  the  same 
question.  Because  the  score  depends  on  the 
tester's  perception  of  "good"  or  "bad,"  the 
test  score  cannot  be  compared  consistently 
with  BESTs  administered  by  other  testers. 
For  true  consistency,  the  BEST  needs  to  be 
administered  in  the  same  way  each  time 


by  the  same  person.  If  the  test  is  being 
used  internally  at  one  agency,  skipping 
one  question  will  not  affect  class  levels 
if  all  of  the  tests  have  the  same  question 
omitted.  However,  because  we  work  with 
other  agencies  and  often  refer  students 
back  and  forth,  test  scores  may  not  be  con- 
sistent with  those  given  at  other  agencies. 

I feel  that  the  BEST  is  appropriate  for 
what  it  was  designed  for:  as  an  intake  tool 
for  those  with  a low  level  of  English.  For 
levels  of  English  higher  than  basic  survival 
skills,  however,  a new  assessment  tool 
needs  to  be  found. 


0 


ERIC 


siTURES  IN  ASSESSMENT 


784 


Volume  10:  December  1997 


Volume 
10  • 73 


What  Counts? 

Assessing  Computer  Skills 


1am  currently  working  as  an  instructor 
at  Malden  Mills  in  Lawrence,  Massa- 
chusetts in  a project  funded  by  the 
Corporation  of  Business,  Work,  and  Learn- 
ing. One  of  my  first  tasks  was  to  design  an 
"Introduction  to  Computers"  class  along 
with  assessment  tools  that  could  help  us 
place  students  at  the  appropriate  level  and 
measure  both  an  individual  s progress  and 
the  effectiveness  of  the  class. 

I wanted  to  develop  a rubric  that  con- 
sisted of  an  even  number  of  competency 
levels,  since  people  have  a tendency  to  go 
for  the  middle  when  there  is  an  odd  num- 
ber. I came  up  with  descriptions  of  six  lev- 
els of  computer  use  (Figure  1)  ranging 
from  never  having  used  a computer  (level 
1)  to  basically  being  a computer  guru  (level 
6).  Our  Introduction  to  Computers  course 
was  designed  to  serve  people  from  levels 
1,2,  and  3.  Each  class  worked  best  and  the 
participants  made  the  most  efficient  use  of 
their  time  if  the  abilities  of  the  participants 
differed  by  no  more  than  one  level.  A class 
that  was  mostly  level  1 could  concentrate 
on  learning  how  to  handle  a mouse  and 
how  to  enter  and  leave  programs  while  a 
. class  that  was  mostly  level  3 could  go  into 
programs  in  more  depth. 

Since  I was  giving  an  assessment  to  as 
many  as  twenty  people  at  once,  and  I am 
not  a particularly  good  detective,  I decided 
to  utilize  self-assessments.  With  each 
group,  I walked  them  through  the  six 
levels  of  the  rubric  in  detail,  expanding 
on  exactly  what  I meant  by  each  level  and 
emphasizing  the  importance  of  answering 
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honestly.  If  a person  was  really  level  3 and 
self-described  herself  as  level  1,  she  might 
be  bored  in  a class  that  spent  an  hour 
learning  to  handle  a mouse  and  another 
hour  learning  to  open  and  close  programs. 

We  decided  a few  weeks  later  that  more 
than  90%  of  the  students  demonstrated 
computer  knowledge  and  performance  in 
class  that  matched  their  self-assessments. 
Since  I also  wanted  some  objective  infor- 
mation, I developed  a short  questionnaire 
to  go  along  with  the  rubric  (Figure  2)  to  get 
a sense  of  both  computer  and  written  lan- 
guage skills.  If  questions  were  answered 
inappropriately  or  not  at  all,  I would  refer 
the  participant  to  our  ESOL  assessment 
process.  If  questions  were  answered  with 
considerable  sophistication,  I might  refer 
the  participant  to  a community  college 
program  more  appropriate  for  an  ad- 
vanced student  than  our  introductory 
course. 

I needed  to  develop  a checklist  of  skills 
that  our  course  would  address.  My  first 
effort  (Figure  3)  drew  upon  many  years 
of  introducing  computers  to  adults  plus  a 
month  of  examining  the  needs  of  workers 
and  managers  at  Malden  Mills  by  going 
on  repeated  tours  of  the  plant  and  having 
numerous  informal  conversations  with 
both  workers  and  managers.  In  addition,  I 
received  the  considerable  assistance  of  my 
colleagues,  Cindy  Cook,  Judy  Hikes,  and 
Johan  Uvin.  While  both  Cindy  and  Judy 
had  extensive  experience  with  adult  stu- 
dents and  had  worked  with  computers  in  a 
classroom  setting  before,  this  was  the  first 
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time  they  were  teaching  a formal  computer 
course.  They  provided  a continual  reality 
check  for  me  and  helped  me  from  becom- 
ing too  technical,  while  I provided  the 
overall  course  outline  and  first  draft  of  the 
assessment  form,  as  well  as  technical  assis- 
tance. The  three  of  us  taught  a total  of  nine 
sections  of  the  Introductory  course.  Two 
more  instructors  have  been  added  in  our 
second  cycle. 

While  the  rubric  gave  us  a general  idea 
of  the  computer  expertise  of  our  partici- 
pants, the  checklist  gave  us  a far  more 
detailed  look  at  what  students  knew.  In 
the  first  cycle,  I gave  the  checklist  to  my 
students  on  the  first  day  of  class  and  had 
them  check  off  the  items  they  knew  under 
the  Pre  column.  But  following  Johan's  sug- 
gestion, we  changed  to  a rating  system  in 
which  1 meant  they  couldn't  do  the  task, 

2 meant  they  could  complete  the  task  with 
some  prompting,  and  3 meant  they  could 
do  the  task  independently  in  a classroom 
situation.  I used  the  new  system  for  the 
first  time  when  I worked  with  students  to 
complete  the  "Post"  column  of  the  check- 
list. Since  I now  knew  the  students'  com- 
puter competencies,  I was  able  to 
participate  in  the  "Post"  test  with  them. 

Our  experience  with  the  first  cycle, 
along  with  getting  some  new  capabilities 
on  the  computers,  led  to  an  expanded  form 
(Figure  4).  We  added  items  that  we  were 
able  to  work  on  in  the  first  cycle,  or  that  we 


decided  we  wanted  to  add  to  the  curricu- 
lum for  the  second  cycle. 

As  an  additional  reality  check  whether 
our  efforts  were  relevant  to  the  needs  of 
the  workers  and  the  workplace,  my  final 
task  was  to  relate  our  Computer  Assess- 
ment Scoring  Guide  with  the  National  Skill 
Standards  Project  for  Advanced  High  Per- 
formance Manufacturing  (Figure  5).  The 
National  Skill  Standards  Project  is  a work- 
in-progress  that  is  trying  to  articulate  those 
skills  needed  by  workers  across  many 
industries.  Such  standards  can  be  used 
as  a tool  to  analyze  the  skills  needed  to 
perform  a certain  job.  They  can  also  pro- 
vide valuable  insight  for  educators  work- 
ing with  future  workers. 

I felt  that  our  Computer  Assessment  ad- 
dressed problem  solving  skills  (PS),  as  well 
as  computer  skills  (CU).  In  general,  I think 
that  determining  how  to  use  a computer 
to  solve  a new  problem  or  complete  a new 
task  requires  greater  understanding  than 
just  learning  how  to  use  a particular  pro- 
gram and  being  instructed  when  to  use  it. 
For  example,  it  takes  a greater  understand- 
ing to  use  a computer  to  troubleshoot  a 
malfunctioning  machine  than  it  does  to  re- 
peatedly go  through  a standard  procedure 
with  a correctly  functioning  machine. 
Finally,  rather  than  being  an  end  in  itself, 
at  the  highest  level  the  computer  becomes 
just  one  more  tool  in  a broader  process  of 
problem  solving. 
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FIGURE  1 


Project  STEPS  Name 

Date 

COMPUTER  ASSESSMENT  SCORING  GUIDE 

1 . Has  never  used  a computer.  Is  very  worried  and  uncomfortable  about  having  to 
use  a computer. 

2.  Has  used  a computer  at  least  a few  times'.  Can  turn  a computer  on,  but  under- 
stands little  else.  Needs  considerable  direction  and  support  to  handle  even  the 
most  fundamental  tasks.  Cannot  handle  a mouse  without  difficulty. 

3.  Used  a computer  competently  in  a structured  work  situation,  but  it  still  uncomfort- 
able with  unfamiliar  applications.  Can  follow  clear  step-by-step  directions,  but 
cannot  use  the  computer  as  a tool.  Can  use  a mouse  to  move  around  the  screen. 
Does  not  fully  understand  how  programs  and  data  differ  and  how  they  are  orga- 
nized, stored,  and  accessed. 

4.  Has  some  experience  using  the  computer  as  a tool.  Can  use  word  processing  and 
other  programs.  Can  learn  additional  programs  with  training.  Is  able  to  experi- 
ment and  try  different  things  out  on  the  computer.  Understands  the  basic  structure 
of  the  computer  and  the  difference  between  programs  and  data. 

5.  Experienced  and  confident  using  the  computer  as  a tool.  Able  to  do  troubleshoot- 
ing to  solve  problems.  Comfortable  using  help  systems  and  manuals  to  learn  how 
to  use  software.  Able  to  determine  the  appropriate  software  to  use  to  accomplish 
a task. 

6.  Frequently  helps  others  with  their  computer  problems.  Can  clearly  explain  to  others 
computer  capabilities  and  functions.  Can  evaluate  software  and  hardware.  Tests 
new  software  and  creatively  uses  it  to  improve  productivity  and  quality. 


Self-assessment  score 
Assessment  score 
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FIGURE  2 


Project  STEPS 


Name 


Date 


INTRODUCTION  TO  COMPUTERS  ASSESSMENT 


1 .  How  do  you  properly  turn  off  a computer? 


2.  Do  you  know  how  to  type? 


3.  What  is  the  difference  between  an  operating  system,  such  as  Windows  95,  and 
an  application  program,  such  as  Microsoft  Word? 


4.  Using  a mouse,  how  can  you  select  something  on  the  screen? 


5.  What  are  you  using  a computer  for  right  now? 


6.  If  you  are  not  currently  using  a computer  on  your  job,  what  do  you  think  you  might 
have  to  use  a computer  for  in  the  next  year? 


Assessment  Score 
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FIGURE  3 

Introduction  to  Computers 

Individual  Record  for:  Class: 


Skill 


Pre 


Post 


Turn  on  computer 
Correctly  turn  off  computer 
Use  a mouse 
Use  a keyboard 

Describe  what  makes  a computer  a computer 
Adjust  monitor 

Start  a program  by  double-clicking  an  icon 

Start  a program  from  the  Program  list  or  Windows  Explorer 

Open  a document 

Save  a document 

Close  a document 

Close  a program 

Understand  the  difference  between  a program  and  a document 

Understand  the  file  structure  of  Windows  95 

Navigate  the  file  structure  of  Windows  95  using  Windows  Explorer 

Create  a Microsoft  Word  document 

Print  a document 

Change  print  options 

Change  screen  options 

Create  a Microsoft  Excel  document 

Troubleshoot  by  checking  electrical  connections 

Install  software 

Open  the  system  unit 
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FIGURE  4 


Introduction  to  Computers 


Individual  Record  for: 


Class: 


Skill 


Pre 


Post 


Turn  on  computer 
Correctly  turn  off  computer 
Use  a mouse 
Use  a keyboard 

Describe  what  makes  a computer  a computer 
Adjust  monitor 

Start  a program  by  double-clicking  an  icon 

Start  a program  from  the  Program  list  or  Windows  Explorer 

Open  a document 

Save  a document 

Close  a document 

Close  a program 

Understand  the  difference  between  a program  and  a document 

Understand  the  file  structure  of  Windows  95 

Navigate  the  file  structure  of  Windows  95  using  Windows  Explorer 

Create  a Microsoft  Word  document 

Print  a document 

Change  print  options 

Change  screen  options 

Create  a Microsoft  Excel  document 

Troubleshoot  by  checking  electrical  connections 

Install  software 

Open  the  system  unit 

Create  a Microsoft  Access  database 

Enter  data  in  a Microsoft  Access  database 

Select  the  correct  software  for  a task 

Use  tutorial  software 

Use  educational  software 

Find  and  respond  to  screen  prompts 

Confident  in  ability  to  learn  computer  skills  and  procedures 

Modify  screen  properties 

Use  digital  camera 

Touch  up  and  print  digital  pictures 

Understand  how  a computer  can  control  production  equipment 
Read  a process  control  computer  screen  (i.e.  Wonderware) 

Use  a computer  to  monitor  and  control  production  equipment 
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FIGURE  4 

Relationship  of  Project  STEPS 
Computer  Assessment  Scoring  Guide 
with  the  National  Skill  Standards  Project 
for  Advanced  High  Performance  Manufacturing 

A worker  who  scores  at  level  1 , 2,  or  3 would  not  meet  any  of  the  National  Skill 
Standards. 

Depending  on  the  programs  learned,  a worker  at  Malden  Mills  functioning  at  level  4 
would  meet  one  or  more  of  the  following  standards: 

GU  3 List  various  methods  of  tracking  inventory. 

CU  5 Demonstrate  use  of  an  industry-accepted  word  processing  software  package. 
CU  6 Demonstrate  use  of  an  industry-accepted  spreadsheet  software  package. 

CU  7 Demonstrate  use  of  an  industry-accepted  database  software  package. 

A worker  at  Malden  Mills  functioning  at  level  5 would  meet  all  the  level  4 standards 
plus: 

CU  1 List  possible  computer  applications  in  manufacturing  processes. 

CU  2 Identify  possible  effects  of  introducing  computers  into  manufacturing  processes. 
PS  2 Apply  a system  of  problem  solving. 

PS  3 Identify  opportunities  for  applying  problem  solving  techniques. 

A worker  at  Malden  Mills  functioning  at  level  6 would  meet  all  the  level  5 standards 
plus: 

PS  1 Explain  the  value  of  applying  a problem  solving  system. 
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Learning  from  Experience 

The  TABE:  Thoughts  from  an  Inquiring  Mind 


Atypical  day  in  my  life  starts  like 
this:  I sit  in  a stack  on  the  book 
shelf.  Suddenly,  I am  whisked 
away  and  handed  to  someone  who  looks 
as  startled  by  me  as  I am  by  them.  I am  a 
TABE  (The  Test  of  Adult  Basic  Education) 
and  I have  become  according  to  one  adult 
education  professional,  the  " industry  stan- 
dard." Lots  of  people  know  my  name.  Some 
love  me  and  some  hate  me.  I guess  you 
can't  please  all  the  people  all  of  the  time. 


by 
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The  TABE  is  a battery  of  multiple- 
choice  tests.  According  to  the  publisher, 
the  purpose  of  the  battery  is  "not  to  test 
specific  life  skills,  but  to  test  basic  skills 
in  the  context  of  life  skills  tasks"  (CTB/ 
McGraw  Hill,  1987).  There  is  a vocabulary 
section  and  a reading  comprehension  sec- 
tion, which  together  give  a composite  read- 
ing score.  A locator  test  is  available  which 
consists  of  25  multiple  choice  items  and  25 
multiple  choice  computation  items  ranging 
from  whole  numbers  skills  to  decimals. 

The  locator  requires  37  minutes  to  admin- 
ister (for  both  vocabulary  and  arithmetic 
sections).  There  are  also  two  math  sections 
and  two  writing  sections. 

Programs  vary  a great  deal  on  which 
sections  (or  how  many  sections)  of  the 
TABE  are  given.  The  reading  section  is  al- 
most always  one  of  the  sections  included. 

You  can't  please  all  the  people  all  the 
time.  The  same  could  be  said  for  any  type 
of  assessment.  The  question  for  me  as  an 
adult  educator  and  staff  development  per- 
son is,  does  this  test  meet  my  needs,  the 


needs  of  my  program,  and  the  needs  of 
my  learners? 

To  begin  to  address  these  questions, 

I look  at  my  own  experience  as  an  adult 
educator.  I have  been  able  to  gain  a fairly 
accurate,  general  idea  of  a learner's  read- 
ing comprehension  level  by  using  the 
TABE.  Someone  might  come  to  my  class 
on  any  given  day  with  a TABE  score  of  5.5, 
for  example.  This  gives  me  an  idea  of 
where  to  start.  It  does  not,  however,  give 
me  an  idea  of  which  materials  I might  try 
with  this  learner. 

Still,  I have  learned  over  the  years  to 
take  that  score  with  a rather  large  grain  of 
salt.  When  I talk  to  my  learners  about  their 
scores  on  the  TABE  (which  they  are  almost 
always  anxious  to  find  out),  I tell  them  that 
this  score  only  gives  us  a ballpark  figure 
and  that  we  will  both  know  better  after  a 
few  weeks  of  working  together,  at  which 
"level,"  for  want  of  a better  word,  they  are. 

We  also  talk  about  the  value  of  knowing 
a " level."  We  discuss  how  it  can  give  us  a 
general  idea  about  how  far  they  might  be 
from  being  able  to  take  the  GED  ( which  is 
very  often  their  first,  and  sometimes  only, 
stated  goal). 

My  career  in  adult  education  had  almost 
always  involved  the  wearing  of  at  least 
two  different  hats.  One  is  that  of  the  Practi- 
tioner Inquiry  Coordinator  (PIC).  In  my 
role  as  PIC,  I work  with  teachers  to  see  the 
worlds  of  our  classrooms  through  the  eyes 
of  an  anthropologist.  We  observe  carefully. 
We  try  to  answer  the  question:  What  is  go- 
ing on  in  this  classroom?  We  try  to  identify 
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and  question  the  underlying  assumptions 
in  our  teaching.  Sometimes  we  make 
changes  based  on  that. 

A counselor  I spoke  with  told  me  that 
the  TABE  is  the  "industry  standard."  I 
asked  how  long  the  program  had  been 
using  the  TABE.  He  told  me  that  they  had 
been  using  the  TABE  "since  1973  when  I 
got  here." 

As  a teacher  who  had  taught  in  a num- 
ber of  different  programs  and  settings,  and 
as  a staff  developer  who  has  contact  with 
many  different  teachers,  I then  considered 
the  question:  How  is  the  TABE  used?  What 
is  it  used  for? 

Many  programs  seem  to  use  the  TABE 
as  an  initial  assessment  tool  to  determine 
placement  in  one  of  three  (typically  ABE, 
Pre-GED,  and  GED)  programs.  Some  pro- 
grams administer  the  TABE  on  a regular 
basis  to  determine  movement  to  higher 
levels. 

Some  funders  do  this  too.  Some  man- 
date intervals  at  which  the  TABE  must  be 
taken.  In  one  such  case,  I had  several  stu- 
dents who  needed  to  take  the  TABE  after 
every  150  hours  of  class  time  (about  every 
two  to  three  months).  Some  of  these  stu- 
dents had  taken  the  same  form  of  the  test  a 
few  times  even  before  they  got  to  my  class. 

Some  students  have  told  me  that  their 
goal  for  class  was  to  reach  an  eighth  grade 
level  on  the  TABE  so  that  they  could  enter 
a particular  job  training  program.  In  some 
cases,  a great  deal  was  riding  on  this  par- 
ticular test  score.  For  these  students,  pass- 
ing the  test  (meaning  scoring  at  the  eighth 
grade  level)  became  their  priority  (under- 
standably so).  So... I learned  rather  quickly 
in  my  career  that  the  TABE  could  some- 
times make  or  break  a student's  potential 
career  path. 
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A study  published  in  1995  by  the  Na- 
tional Center  for  Adult  Literacy  ( NCAL) 
entitled  "When  less  is  more:  A comparative 
Analysis  for  Placing  Students  in  Adult  Lit- 
eracy Classes"  concluded  that  "a  test  as 
brief  as  the  TABE  locator  could  predict 
placements  as  well  as  the  complete  group 
of  reading  tests"  The  following  sums  up 
their  recommendations: 

"Attempts  to  achieve  extremely  high 
accuracy  in  placement  should  be  tempered 
by  a consideration  of  the  small  number  of 
placement  levels  usually  available....  Over- 
all it  may  be  concluded  that  less  testing 
may  be  more  valuable  to  both  students  and 
adult  literacy  programs.  Less  time  on  test- 
ing means  less  cost  for  testing.  Perhaps 
more  importantly,  learners  often  have  dis- 
taste for  and  fear  of  standardized  tests.  By 
cutting  back  on  testing  and  moving  toward 
a self  assessment  model,  programs  may 
stimulate  greater  motivation  and  satisfac- 
tion among  the  clients  they  serve." 

Based  on  my  experience,  I would  recom- 
mend we  consider  the  following  questions: 

• When  and  why  did  we  all  decide  that 
the  TABE  was  the  " industry  standard"? 

• Does  the  TABE  help  us  find  out  the 
information  we  are  seeking  to  know? 

• What  do  we  seek  to  know  from  using 
the  TABE? 

• To  what  extent  is  the  TABE  successful 
in  placing  students  in  the  correct  classes? 

• Is  there  flexibility  in  our  programs  when 
the  TABE  results  are  not  successful  in  plac- 
ing students  in  the  correct  classes? 

• Are  we  using  the  TABE  in  a way  that  is 
consistent  with  the  intended  purposes  of 
the  test? 

• Does  the  TABE  help  learners  identify 
needs  and/or  levels? 

It  is  possible  that  the  TABE  is  indeed  the 
very  best  test  to  use  to  determine  this  kind 
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of  information.  If  we  take  an  inquiry  ap- 
proach to  this  issue,  however,  and  examine 
the  underlying  assumptions , we  may  dis- 
cover important  information  that  can  help 
us  all  better  assess  the  needs  of  our  learn- 
ers and  our  programs. 


If  so  much  is  going  to  ride  on  the  results 
of  a standardized  test,  perhaps  we  should 
take  a moment  to  step  back  and  think 
about  the  purpose  of  a standardized  test, 
what  it  can  and  cannot  tell  us  and  if  indeed 
this  is  the  most  appropriate  test  to  use. 

Inquiring  minds  want  to  know. 
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Review: 

Using  Phenomenal  Changes  - Stories  of 
Participants  in  the  Portfolio  Project 


k 


Ihave  been  thinking  a lot  about  com- 
ing up  with  a better  model  of  portfolio 
assessment  in  our  school  so  we  can  be 
more  consistent  with  the  process.  In  the 
past,  I would  always  come  to  the  same 
wall  that  I have  with  authentic  assessment: 
you  can't  force  people  to  do  it  if  they  don't 
have  a full  understanding  of  the  process. 

To  truly  implement  any  type  of  authentic 
assessment,  people  need  to  buy  into  the 
concept. 

But  how  do  you  accomplish  that  when 
there  just  doesn't  seem  to  be  enough  time 
to  nurture  the  process?  It  seemed  seren- 
dipitous that  I was  asked  to  review  Phe- 
nomenal Changes  - Stories  of  Participants  in 
the  Portfolio  Project  for  Adventures  in  Assess- 
ment as  it  allowed  me  to  focus  on  how  to 
get  both  me  and  the  teachers  at  the  Inter- 
national Language  Institute  (ILI)  of  MA  in 
Northampton  more  involved  in  improving 
how  we  do  portfolio  assessment  at  the 
school. 

(This  is  the  first  of  a two-part  article  as 
we  won't  finish  our  project  until  the  next 
Adventures  in  Assessment  is  out;  we  all 
know  that  Authentic  Assessment  is  an 
ongoing  project.) 

Phenomenal  Changes  - Stories  of  Partici- 
pants in  the  Portfolio  Project  is  an  excellent 
staff  development  tool  for  portfolio  devel- 
opment. At  the  ILI,  we  are  using  the  text  as 
a springboard  for  discussions  about  port- 
folio assessment. 

The  articles  in  Phenomenal  Changes  — a 
series  of  interviews  with  the  participants  of 
the  project  and  the  Project  Director  Melody 
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Schneider  — are  very  reader  friendly.  The 
text  is  divided  into  five  areas:  ABE,  ESOL, 
Literacy  Councils,  Compensatory  Educa- 
tion, and  Administrators. 

We  have  been  using  several  different 
variations  of  authentic  assessment  tools  at 
the  International  Language  Institute  over 
the  years.  At  a recent  staff  meeting  I asked 
the  teachers  how  it  was  going  with  portfo- 
lio assessment  in  their  classes.  The  teachers 
replied  that  they  have  the  students  using 
dialogue  journals  and  learning  journals 
and  that  they  had  been  collecting  things, 
but  they  didn't  really  feel  as  if  they  had  a 
handle  on  the  concept  of  portfolios.  We 
agreed  portfolios  could  take  many  forms, 
but  that  the  group  needed  a model  with 
which  we  all  felt  comfortable.  I brought 
out  Phenomenal  Changes  and  asked  if  they 
would  be  interested  in  reading  about  the 
experiences  of  other  teachers  working  on 
portfolio  assessment.  They  were  very  inter- 
ested, so  we  agreed  to  read  and  discuss  ar- 
ticles for  the  next  five  weeks.  During 
curriculum  week  at  the  end  of  the  program 
we  would  put  together  all  our  ideas  and 
develop  criteria  for  portfolio  selection  and 
review.  We  plan  to  put  our  findings  into 
place  for  the  program  that  begins  in  Janu- 
ary. 

We  decided  to  begin  our  weekly  meet- 
ings with  30  minutes  dedicated  to  portfolio 
assessment  discussion.  (Weekly  meetings 
usually  last  an  hour  and  a half  and  include 
both  part-time  and  full-time  staff.)  The  30- 
minute  segment  would  be  divided  into 
three  steps.  First,  we  would  discuss  our 
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reactions  to  an  article.  Second,  we  would 
discuss  how  this  article  would/wouldn't 
relate  to  our  program.  Third,  we  would 
establish  some  action  plans  to  implement 
in  the  next  month  or  during  curriculum 
week.  Since  the  staff  are  ESL  teachers,  we 
started  off  with  the  two  articles  in  the  text 
that  discussed  portfolio  assessment  with 
ESL  teachers. 

The  first  comment  that  came  up  the  fol- 
lowing week  at  our  first  discussion  was 
that  everyone  would  have  preferred  the 
articles  to  be  actually  written  by  the  writer 
and  that  vernacular  usage  was  a bit  diffi- 
cult to  read.  We  agreed  that  Melody 
Schneider  does  a good  job  of  letting  the 
reader  hear  the  project  participant's  voice. 

One  teacher  said  she  would  have  liked 
more  information  about  the  contents  of  a 
portfolio  and  more  nuts  and  bolts  about 
the  end  product.  A discussion  ensued 
about  how  we  could  look  at  lists  that  have 
been  prepared  by  other  people,  but  would 
that  help  find  our  own  way  of  doing  port- 
folio assessment?  We  agreed  that  not  hav- 
ing the  "nuts  and  bolts"  up  front  enabled 
us  to  look  at  portfolio  assessment  without 
being  directed  as  to  what  we  should  be  do- 
ing. This  discussion  led  to  asking,  'what 
should  we  be  putting  in  the  portfolios?' 
One  teacher  said  that,  since  her  students 
were  only  in  class  six  hours  a week,  how 
about  including  things  that  the  students 
accomplished  outside  class? 

Another  teacher  commented  that  the 
articles  really  helped  her  realize  that  she 
needs  to  get  out  of  her  students'  way.  She 
felt  that  she  was  holding  their  hands  too 
much  and  that  they  needed  to  "experi- 
ence" something  rather  than  having  her 
always  telling  them  what  to  do. 


We  also  talked  about  the  dialogue  jour- 
nals that  our  students  keep.  Some  students 
write  a lot  and  "play"  with  the  language, 
using  what  they  have  learned  in  class 
while  others  write  the  same  thing  every 
week:  'I  went  to  church  and  cleaned  my 
house.'  How  can  we  show  students  their 
progress  if  they  write  the  same  things 
week  after  week? 

Our  discussion  led  to  Considering  ways 
students  can  see  their  progress  through 
guided  writing,  writing  with  a specific 
task,  and  making  the  writing  process  inter- 
esting for  students.  We  also  ended  up 
sharing  a writing  tool  called  Fundex  of  In- 
dividualized Activities  for  English  Lan- 
guage Practice*  that  the  teachers  weren't 
previously  aware  of  and  became  very  ex- 
cited to  discover. 

The  teachers  acknowledge  that  both  stu- 
dents and  teachers  need  to  understand 
why  we're  doing  portfolio  assessment.  "I 
need  to  understand  why  we're  doing  this. 
How  do  I encourage  it  to  be  taken  seri- 
ously? How  do  I motivate?"  We  admitted 
that  we  as  teachers  should  also  be  keeping 
portfolios  about  our  teaching  practices. 
Included  in  our  portfolios  would  by  our 
own  Learning  Journals  (a  booklet  of  week- 
ly assessment  tools  kept  by  the  students 
(see  Adventures  in  Assessment,  Volume  5, 
"Evolution  of  an  Assessment  Tool").  These 
would  be  completed  in  class  while  our  stu- 
dents were  filling  out  their  own  Learning 
Journals.  One  teacher  said  her  students 
might  take  portfolio  assessment  more  seri- 
ously if  they  saw  her  taking  it  seriously. 

Naturally,  we  went  over  the  allotted 
30  minutes.  At  the  end  of  the  meeting  we 
did  feedback  about  using  the  articles  as  a 
springboard  for  discussion  and  all  the 


* Writing  Inspirations:  A Fundex  of  Individualized  Activities  for  English  Language  Practice  by  Arlene  Marcus, 
Prolingua  Asscoiates,  15  Elm  Street,  Brattleboro,  VT  05301,  (802)  257-7779. 
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teachers  were  enthusiastic:  "There  were  a 
lot  of  things  that  I got  from  these  articles," 
"I  like  making  time  to  do  this,"  and  "It's 
kind  of  nice  to  have  an  academic  focus  in 
our  meeting,  rather  than  just  going  over 
the  attendance  list." 

As  I listened  and  participated  in  the 
meeting,  I realized  that  we  are  all  stake- 
holders in  the  process  and  that  we  have 
to  realize  that  authentic  assessment  is  a 
change  of  practice  for  everyone:  teachers, 
students  and  administrators. 

I also  realized  that  a variety  of  people 
interested  in  portfolio  assessment  could 
use  Phenomenal  Changes  - Stories  of  Partici- 


pants in  the  Portfolio  Project.  The  readings 
are  just  as  useful  for  people  just  starting 
out  as  for  people  that  have  been  doing  it 
for  awhile. 

We  left  our  meeting  excited  and  armed 
with  two  more  articles  for  the  next  week. 
We  don't  know  exactly  how  our  portfolio 
system  will  be  in  January,  but  at  this  point 
I feel  strongly  that  with  the  consistency 
of  the  weekly  meetings  about  portfolio 
assessment  and  using  Phenomenal  Changes, 
there  will  be  some  phenomenal  changes 
of  how  we  all  look  at  portfolio  assessment 
at  ILL 

to  be  coniinueu 
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Phenomenal  Changes:  Stories  of  Participants  in  the  Portfolio  Project,  Melody  Schneider,  Project  Director,  with 
Hanna  Fingeret  and  Loren  McGrail,  Literacy  South  available  from  Peppercorn  Books,  P.O.  Box  1766, 
Durham,  NC  27702,  (919)  688-9313. 


o 

ERIC 

UBiPj lame  10:  December  1997 


797 


ADVENTURES  IN  ASSESSMENT 


& 


approach® 


< 


to 


M 


evaluatie 


o 

ERIC 


S A B E S 

Sydam  tor  Adbft  Buie  Educaion  Support 

A 


SABES  is  the  System  for  Adult  Basic  Education  Support,  a comprehensive  training  and 
technical  assistance  initiative  for  adult  literacy  educators  and  programs.  Its  goal  is  to 
strengthen  and  enhance  literacy  services  and  thus  to  enable  adult  learners  to  attain  literacy 

skilb. 


SABES  accomplishes  this  goal  through  staff  and  program  development  workshops,  consulta- 
tion, mini-courses,  mentoring  and  peer  coaching,  and  other  training  activities  provided  by 
five  Regional  Support  Centers  located  at  community  colleges  throughout  Massachusetts. 
SABES  also  offers  a 15-hour  Orientation  that  introduces  new  staff  to  adult  education  theory 
and  practice  and  enables  them  to  build  support  networks.  Visit  us  at  our  website: 
www.sabes.org 


SABES  also  maintains  an  adult  literacy  Clearinghouse  to  collect,  evaluate,  and  disseminate 
ABE  materials,  curricula,  methodologies,  and  program  models,  and  encourages  the  develop- 
ment and  use  of  practitioner  and  learner-generated  materials.  Each  of  the  five  SABES 
Regional  Support  Centers  similarly  offers  program  support  and  a lending  library.  SABES 
maintains  an  Adult  Literacy  Hotline,  a statewide  referral  service  which  responds  to  calls 
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The  SABES  Central  Resource  Center,  a program  of  World  Education,  publishes  a statewide 
quarterly  newsletter,  "Bright  Ideas,"  and  journals  on  topics  of  interest  to  adult  literacy 
professionals,  such  as  this  volume  of  "Adventures  in  Assessment." 

The  first  three  volumes  of  "Adventures  in  Assessment"  present  a comprehensive  view  of  the 
state  of  practice  in  Massachusetts  through  articles  written  by  adult  literacy  practitioners. 
Volume  1,  Getting  Started,  includes  start-up  and  intake  activities;  Volume  2,  Ongoing, 
shares  tools  for  ongoing  assessment  as  part  of  the  learning  process;  Volume  3,  Looking 
Back,  Starting  Again,  focuses  on  tools  and  procedures  used  at  the  end  of  a cycle  or  term, 
including  self , class,  and  group  evaluation  by  both  teachers  and  learners.  Volume  4 covered 
a range  of  interests,  and  Volume  5;  The  Tale  of  the  Tools  is  dedicated  to  reflecting  on 
Component  3 tools  of  alternative  assessment.  Volume  6,  Responding  to  the  Dream  Con- 
ference, is  dedicated  to  responses  to  Volumes  1-5.  Volume  7,  The  Partnership  Project, 
highlighted  writings  from  a mentoring  project  for  practioners  interested  in  learning  about 
participatory  assessment.  Volumes  8-11  cover  a range  of  topics,  including  education  reform, 
workplace  education,  learner  involvement  in  assessment,  etc. 

We'd  like  to  see  your  contribution.  If  you  would  like  to  submit  an  article  for  our  Winter  1999 
issue,  contact  Editor  Alison  Simmons. 


Opinions  expressed  in  "Adventures  in  Assessment"  are  those  of  the  authors  and  not  necessarily 
the  opinions  of  SABES  or  its  funders. 

Permission  is  granted  to  reproduce  portions  of  this  journal;  we  request,  however,  appropriate 
credit  be  given  to  the  author  and  to  Adventures  in  Assessment. 

Adventures  in  Assessment  is  free  to  DOE-funded  Massachusetts  programs;  out-of-state 
requests  will  be  charged  a nominal  fee.  Please  write  to,  or  call: 


Alison  Simmons  or  Catherine  Santiago 
World  Education 
44  Farnsworth  Street 
Boston,  MA  02210-1211 
617-482-9485 


asimmons@worlded.org 

csantiago@worlded.org 
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Introduction 

Volume  11:  Aspects,  Levels, 
and  Perspectives 


by 

Alison  Simmons 


Editor 

SABES  Central 
Resource  Center/ 
World  Education 


As  a staff  developer  and  a teacher,  as- 
sessment remains  a daunting  pros- 
pect for  me.  I am  often  overwhelmed 
with  the  amount  of  knowledge  and  skills 
that  I need  to  further  my  work  in  this  area. 

I must  get  to  a place  where  I feel  comfort- 
able with  the  choices  that  have  been  made 
about  assessment  practices,  both  for  my 
students  and  for  other  teachers. 

I want  to  ensure  that  I am  providing 
opportunities  for  students  to  help  them 
identify  what  it  is  they  want  to  learn  and 
opportunities  to  see  that  learning.  I also 
want  to  provide  opportunities  for  teachers 
to  get  the  information  they  need  to  make 
more  informed  decisions  about  their  as- 
sessment practices,  both  in  their  class- 
rooms and  in  their  programs. 

There  are  many  different  perspectives 
on  assessment,  accountability,  and  evalua- 
tion. Some  connect  and  others  seem  miles 
apart.  In  order  for  me  to  make  sense  of  the 
information  and  the  differences  in  perspec- 
tives and  to  make  informed  choices,  I need 
to  be  able  to  look  at  the  whole  and  try  to 
see  where  the  connections  are. 

Often  we  are  engrossed  in  one  aspect 
of  assessment,  usually  the  one  that  affects 
us  the  most.  This  often  depends  largely  on 
the  role(s)  we  play  in  adult  education.  We 
need  to  be  informed  and  aware  of  the  other 
aspects,  perspectives  and  levels  of  assess- 
ment in  our  work.  We  need  to  see  how 
they  can  fit  together  and  how  each  per- 
spective and  level  tries  to  answer  the  ques- 
tions Are  the  adults  in  our  programs 
receiving  what  they  need?  Are  they  mak- 
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ing  progress  towards  their  goals  and  are 
our  programs  providing  opportunities  for 
this  to  happen?  We  need  to  look  at  the 
various  levels  of  assessment  and  account- 
ability — local  and  state  and  federal  — and 
negotiate  between  them"  (Heide  Spruck 
Wrigley,  AiA,  Volume  11). 

When  I read  the  articles  for  this  volume 
of  Adventures  in  Assessment,  I am  much  less 
daunted  by  the  ongoing  task  of  trying  to 
"negotiate"  and  understand  the  different 
levels  mentioned  above.  The  articles  in  this 
issue  seem  to  offer  opportunities  to  think 
of  different  aspects  of  assessment.  They 
raise  questions  not  just  about  a specific 
tool,  but  also  how  assessment  is  connected 
to  teaching  and  learning  and  how  it  can  be 
connected  to  accountability  for  programs 
as  well  as  teachers. 

I do  not  think  the  articles  solve  the  ques- 
tions of  understanding,  choosing,  report- 
ing gains,  and  promoting  assessment 
practices,  but  I feel  less  anxious.  Through 
the  articles  I feel  we  are  moving  closer  to 
an  understanding  or  an  awareness  of  how 
and  what  to  negotiate  in  the  world  of  as- 
sessment. 

Marie  Hassat's  article  looks  at  assess- 
ment and  curriculum  development.  She 
has  adapted  a framework  for  teaching  by 
Herbert  Kohl  and  talks  about  how  she 
adapted  this  to  her  program  and  assess- 
ment practices. 

Caroline  Gear  and  Joanna  Scott  move 
away  from  assessment  for  students'  progress 
and  ask  us  to  look  at  what  we  as  teachers 
do  in  our  classrooms  to  assess  and  evalu- 
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ate  our  lessons  and  classes.  Both  provide  a 
framework  for  reflecting  on  our  lessons.  It 
helps  us  to  look  at  our  teaching  to  see  if  we 
really  are  providing  opportunities  for  stu- 
dents to  learn  and  to  attain  their  goals. 

In  her  article,  Linda  Gosselin  reviews 
the  many  tools  and  ideas  that  have  been 
presented  in  Adventures  in  Assessment  over 
the  years.  She  asks  the  questions  of  learner 
centered  approaches  to  assessment:  What 
does  this  really  mean?  If  we  are  doing  it, 
what  does  this  really  look  like?  When  do 
we  involve  students  and  how?  She  invites 
us  to  critically  assess  our  own  practices 
and  the  roles  we  ask  students  to  play. 

Glen  Cotton  and  Cheryl  Grant  from 
North  Carolina  discuss  their  action  research 
projects.  They  looked  at  the  question  of  as- 
sessment in  ESOL  classes  and  reflected  on 
their  own  practices.  They  both  hint  at  the 
notion  that  assessment  should  not  be  sepa- 
rate from  instruction.  They  state  that  there 
are  ways  for  even  the  beginning  language 
learner  to  get  information  that  helps  set  a 
course  for  instruction  as  well  as  elicits 
information  about  the  learners'  goals  and 
progress. 

Maria  Elena  Gonzalez  takes  us  on  a 
journey  to  the  world  of  assessment  tools. 
She  mirrors  the  process  most  of  us  go 
through  when  trying  to  select  a tool.  She 
realizes  that  there  are  many  tools  available 
and,  although  we  need  to  consider  the 
many  levels  of  assessment,  we  need  also  to 
consider  the  other  factors  that  are  involved 
in  teaching  adults.  Most  importantly,  we 
must  constantly  ask  ourselves  "What  do  I 
really  need  to  know?"  This  helps  narrow 
our  search,  and  keeps  us  grounded  in  what 
is  really  important  for  both  the  teacher  and 
the  students. 

Janet  Isserlis  talks  about  actual  learning 
and  accountability  and  sees  the  need  for 
hr>th.  They  may  not  be  mutually  exclusive 
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but  we  are  often  driven  to  choose  between 
the  two.  She  also  talks  about  the  need  for 
teachers  to  make  connections  between 
assessment  and  learning.  The  connections 
need  to  be  an  ongoing  part  of  the  staff 
development  that  we  provide.  "We  need 
to  know  what  connecting  assessment  to 
learning  looks  like  and  how  to  help  learn- 
ers see  their  own  progress  through  mul- 
tiple lenses"  (Janet  Isserlis,  AiA,  Volume  11). 

Heide  Spruck  Wrigley  observes  assess- 
ment at  the  federal  and  state  levels.  Where 
do  we  go  from  here?  She  offers  a much- 
needed,  hopeful  perspective  dial  looks  at 
all  levels  of  assessment  and  negotiating 
among  funders,  programs  and  teachers. 

In  Learning  from  Experience , Diane  Lizotte 
talks  about  her  action  research  project.  She 
looked  at  the  strengths  and  weaknesses  of 
the  ESOLA  test  and  found  that  the  test  was  a 
good  tool  for  placement  but  not  for  measur- 
ing learner  progress  over  time. 

The  teachers  at  the  ACCCESS  program 
in  Hyannis,  Massachusetts,  reviewed  the 
book  "New  Ways  in  Assessment/'  published 
by  TESOL.  They  sampled  a few  of  the  ac- 
tivities in  the  book  in  their  classroom.  They 
found  that  the  assessment  activities  were 
engaging  for  students  and  teachers  and 
that  the  book  had  a lot  of  good  ideas  to  try. 

Many  questions  arise  as  we  continue 
our  journey,  searching  for  authentic  ways 
to  assess  students,  programs,  teachers,  and 
ourselves.  The  articles  in  this  issue  direct 
readers  to  view  the  overall  picture  of  as- 
sessment and  reflect  on  the  ever-present 
and  pared-down  question,  "How  could 
this  work  in  my  program?" 

As  always  I invite  readers  to  respond  to 
these  articles  and  to  maintain  the  dialogue 
of  what  works  and  doesn't  work  in  the 
process  of  using  authentic  assessment  in 
our  programs. 

f — Alison  Simmons 
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Evaluation  that  Looks  at 

Realistically 


Every  teacher  tells  stories  about  the 
students  whose  spectacular  suc- 
cesses or  gut-wrenching  defeats 
help  us  to  define  what  we  do.  Less  easy 
to  articulate  are  the  stories  of  the  students 
whose  experiences  we  return  to  again 
and  again,  wondering,  "What  if?"  or 
"If  only..."  as  we  try  to  figure  out  how 
we  could  have  helped  them  more. 

This  is  especially  true  for  adult  educa- 
tors. The  structure  of  our  work  and  the 
needs  of  our  students  force  us  to  focus 
on  the  end  result:  passing  the  GED  exam, 
moving  into  a higher  level  class,  becoming 
a U.S.  citizen.  But  as  we  all  know,  the  dedi- 
cation and  hard  work  of  students  and 
teachers  do  not  always  lead  to  the  achieve- 
ment of  these  goals.  Inadequate  child  care, 
medical  problems,  housing  crises,  and 
other  real-life  challenges  can  undermine 
the  progress  of  even  the  most  determined 
students,  in  the  classroom  of  the  most  sup- 
portive teacher. 

After  facing  this  problem  in  my  own 
classrooms,  I began  searching  for  a tool 
that  would  allow  me  to  deal  with  the  stop- 
in,  stop-out  nature  of  adult  education  more 
productively.  I wanted  an  evaluative  tool 
that  would  allow  me  to  quantify  the 
progress  made  by  students  who  left  before 
passing  the  GED  exam  or  increasing  their 
skills  by  a grade  level;  address  more  spe- 
cifically the  kinds  of  qualitative  learning 
that  do  not  appear  in  an  English  or  a social 
studies  curriculum;  and  give  students^ 
way  to  see  that  they  have  learned,  even  if 
they  have  not  yet  reached  their  target  goals. 
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In  the  course  of  reading  and  research, 

I found  a framework  by  Herbert  Kohl. 
Kohl's  teaching  career  began  in  the 
Harlem  public  schools  in  1962;  his  first 
book,  36  Children,  is  an  account  of  his 
work  with  that  first  class.  During  his  ca- 
reer as  a teacher,  he  has  worked  in  a wide 
variety  of  traditional  and  non-traditionai 
settings,  with  students  who  range  in  age 
from  kindergarteners  to  adults.  As  a pro- 
lific writer  and  researcher.  Kohl  has  fo- 
cused primarily  on  the  ways  that  schools 
sometimes  hamper,  rather  than  enhance, 
the  learning  process  of  many  students, 
and  suggests  a variety  of  ways  for  teachers 
to  help  students  who  are,  in  his  words, 
"creatively  maladjusted"  to  a system  that 
has  been  a source  of  frustration  and  hu- 
miliation. In  his  most  recent  book.  The  Dis- 
cipline of  Hope:  Learning  from  a Lifetime  of 
Teaching,  Kohl  declares  that  "there  are  at 
least  six  basic  skills,  which  encompass  all 
the  trivial  mechanical  skills  that  people 
want  obedient  and  passive  [students]  to 
acquire"  and  which,  if  used  as  a frame- 
work for  curriculum  and  evaluation, 
"respects  the  intelligence  and  moral  sensi- 
bility" of  students  (1998, 234-35).  It  is  this 
framework  which  I have  adopted  for  my 
classes,  and  which  I believe  has  allowed 
me  not  only  to  measure  more  accurately 
the  genuine  achievements  of  my  students, 
but  has  also  helped  me  to  think  more  care- 
fully about  course  content  and  structure. 
The  rest  of  this  article  will  outline  the  six 
skills  Kohl  has  identified,  the  ways  that  I 
have  used  them  with  my  students,  and 
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suggestions  for  implementation  in  others' 
classrooms. 


None  of  the  lists 
of  questions 
presented  here 
are  exhaustive  or 
authoritative; 
one  of  the  real 
strengths  of  Kohl's 
framework  lies  in 
its  flexibility. 


HERBERT  KOHL'S  SIX  BASIC  SKILLS 

Skill  1:  The  ability  to  use  language  well 
and  thoughtfully. 

This  skill  has  obvious  application  for 
teachers  whose  work  focuses  largely  on  lit- 
eracy, but  it  also  presents  opportunities  for 
re-thinking  our  methods  and  goals.  We 
have  all  done  what  we  could  to  help  stu- 
dents use  language  well,  but  I think  I am 
not  the  only  teacher  whose  classes  have 
been,  on  occasion,  less  than  thoughtful.  But 
if  we  think  of  literacy  skills  as  encompass- 
ing what  are  often  referred  to  as  the  four 
language  arts — reading,  writing,  speaking, 
and  listening — this  first  skill  can  help  us  to 
look  at  our  work  more  closely. 

Some  of  the  questions  I've  used  for  evalu- 
ating the  development  of  language  use  in- 
clude: 

• Has  the  student's  vocabulary  increased? 

• Does  the  student  read  more  willingly, 
from  a wider  variety  of  materials?  Have 
I helped  the  student  to  see  reading  as  a 
source  of  information  and  pleasure,  and 
not  just  as  another  requirement? 

• Has  the  student's  writing  become  a 
more  accurate  reflection  of  what  he/ she 
intended  to  say?  Has  the  number  of  me- 
chanical errors  decreased?  Is  the  student 
more  comfortable  as  a writer? 

• Has  the  student's  skill  in  oral  discus- 
sion/ debate  improved?  Can  he/ she 
construct  an  argument  that  relies  on 
logic  more  than  opinion  or  volume? 

None  of  the  lists  of  questions  presented 
here  are  exhaustive  or  authoritative;  one  of 
the  real  strengths  of  Kohl's  framework  lies 
in  its  flexibility.  These  are  the  kinds  of 
questions  I use  when  I evaluate  my  GED 
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and  Basics  II  classes.  An  ESOL  teacher  will 
probably  come  up  with  different  ways  of 
looking  at  this  skill.  This  particular  way  of 
looking  at  skills  has  value  only  when 
teachers  tailor  it  to  the  strengths  and  chal- 
lenges of  their  individual  students  and 
classes. 

Skill  2:  The  ability  to  think  through  a 
problem  and  experiment  with  solutions. 

This  may  be  one  of  the  most  difficult 
skills  for  students  to  master,  and  one  of  the 
most  important.  Frequently,  students  who 
lack  confidence  about  their  ability  become 
nervous  and  flustered  when  problems  are 
presented  in  new  ways.  Because  it  is  im- 
possible to  predict  the  form  of  every  prob- 
lem, question,  or  assignment  they  will  face 
in  their  educational  careers,  it  is  important 
to  help  them  make  connections  between 
diverse  problems  and  let  them  see  that 
they  have  the  ability  to  take  on  new  chal- 
lenges. This  is  also  the  skill  that  helps  stu- 
dents see  the  application  of  various 
class-based  skills  in  real  life  situations.  The 
person  who  can  estimate  while  working  on 
math  problems  can  estimate  in  the  grocery 
store.  A person  who  can  write  a 200-word 
essay  can  write  a cover  letter  for  a job  ap- 
plication. 

The  questions  I use  for  evaluating  this 
skill  include: 

• Has  the  student's  tolerance  for  ambigu- 
ity increased?  Can  he /she  stick  with  a 
problem  until  it  becomes  clear,  or  does 
he/ she  give  up  easily? 

• Can  the  student  see  points  of  common- 
ality between  and  among  different 
things? 

• Can  the  student  understand  the  differ- 
ence between  the  problem  and  the  way 
the  problem  has  been  presented? 
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• Can  the  student  judge  the  relative  merit 
of  several  different  solutions  to  the  same 
problem? 

• Does  the  student  see  how  context  and 
culture  affect  our  ideas  about  what  the 
right  answer  is  to  a given  question? 

This  is  often  the  most  difficult  part  of 
student's  learning  to  assess.  Many  teachers 
have  been  trained,  or  have  received  their 
own  education,  in  an  environment  that 
emphasizes  coverage  of  content  over  com- 
prehension of  process.  Assessing  this  skill 
requires  teachers  to  spend  more  tune  lis- 
tening than  talking,  more  time  following 
students'  ideas  than  presenting  informa- 
tion. Class  discussions  provide  an  oppor- 
tunity for  developing  this  flexibility  of 
thinking,  as  they  allow  exposure  to  a vari- 
ety of  viewpoints,  and  may  demonstrate 
more  clearly  than  any  lecture  that  there  is 
often  more  than  one  way  to  approach  a 
problem.  Students  who  can  be  flexible  in 
their  approach  to  problems  are  more  likely 
to  achieve  greater  success  than  their  less- 
flexible  classmates;  it  is  in  their  best  inter- 
ests that  we  all  find  ways  to  help  them 
develop  this  skill. 

Skill  3:  The  ability  to  understand  scien- 
tific and  technological  ideas  and  to  use 
tools. 

This  skill  has  more,  and  more  vocal, 
proponents  across  the  educational  spec- 
trum than  any  other  on  this  list,  but  again, 
it  is  the  way  that  Kohl  has  framed  the  skill 
that  makes  it  so  valuable.  We  may  cover  a 
variety  of  scientific /technological  topics  in 
class,  but  we  do  not  always  provide  oppor- 
tunities for  experimentation,  practice  and 
reflection.  Many  students  use  sophisticated 
technologies  without  a second  thought — 
VCRs,  beepers,  cell  phones,  ATMs,  and 
video  games — but  few  of  them  have  re- 
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fleeted  on  the  ways  that  those  technologies 
relate  to  the  kinds  of  scientific  and  techno- 
logical material  we  cover  in  our  classes.  By 
linking  understanding  and  use.  Kohl  in- 
vites teachers  to  make  explicit  connections 
between  the  classroom  and  the  real  world. 

It  should  also  lead  teachers  to  reflect  on 
their  own  relationships  with,  and  values 
concerning,  science  and  technology  as  they 
are  a part  of  daily  life. 

Good  questions  about  this  skill  might 
include: 

• What  is  the  student's  comfort  level  with 
computers? 

• Does  the  student  understand  how  tech- 
nology affects  his/her  daily  life? 

• Can  the  student  understand  essays/ ar- 
ticles in  the  popular  press  about  science 
and  technology  issues? 

• Does  the  student  recognize  the  ways 
that  science  has  changed  our  ideas  about 
the  quality  and  length  of  human  life? 

The  questions  a teacher  chooses  for  this 
skill  will  vary  widely  according  to  the 
goals  and  backgrounds  of  students.  Par- 
ticularly with  respect  to  medical  technol- 
ogy, discussions  of  these  topics  address  not 
only  science,  but  values  and  ethics.  I have 
found  that  pursuing  questions  like  this  of- 
ten creates  natural,  powerful  links  between 
the  different  skills  and  content  areas  stu- 
dents are  trying  to  master;  these  are  excel- 
lent issues  to  write  and  debate  about. 

Skill  4:  The  ability  to  use  the  imagina- 
tion. 

Using  one's  imagination  is  a little  ad- 
dressed but  crucial  element  of  academic 
success.  Students  who  can  use  their  imagi- 
nations constructively  are  at  a real  advan- 
tage in  their  reading,  their  writing,  and 
their  ability  to  understand  new  material. 
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Like  the  ability  to  think  through  problems 
and  experiment  with  solutions  (Skill  2),  the 
ability  to  use  one's  imagination  creates  a 
necessary  level  of  flexibility.  Without  it, 
much  of  literature,  film,  science,  and  math- 
ematics will  be  beyond  reach.  But  this  is 
also  a skill  that  most  teachers,  consciously 
or  not,  attempt  to  develop  in  every  class. 
Each  time  we  begin  with  "Let's  suppose," 
"Hypothetically,"  or  "Could  you  suggest 
a situation  where  . . we  are  helping  stu- 
dents learn  to  imagine  in  specific  and  use- 
ful ways. 

Some  of  the  questions  I've  used  for  this 
skill  are  as  follows: 

• Can  the  student  empathize  with  another 
person  or  fictional  character? 

• Can  the  student  come  up  with  alternate 
endings  to  a story? 

• Does  the  student  believe  that  he/she  has 
the  ability  to  transcend  current  limita- 
tions? 

• Does  the  student  understand  the  use  of 
metaphor  and  symbol  in  film,  fiction, 
etc.? 

• Can  the  student  come  up  with  innova- 
tive solutions  to  problems? 

• Does  the  student  think  through  poten- 
tial situations  before  they  occur,  in  order 
to  plan  how  to  respond? 

Most  of  us,  of  course,  do  all  these  things 
without  thinking  about  them.  But  making 
these  skills  explicit  and  concrete  for  stu- 
dents helps  them  see  how  the  skills  they've 
learned  in  their  day-to-day  lives  can  be  of 
value  to  them  in  the  classroom,  and  vice  versa. 

Skill  5:  The  ability  to  understand  how 
people  function  in  groups. 

Many  students  in  adult  education 
classes  lack  formal  and  informal  experi- 
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ence  with  structures  and  institutions  that 
teachers  take  for  granted.  Much  of  their 
experience  with  formal  institutions  may,  in 
fact,  be  negative,  whether  the  institutions 
in  question  are  schools,  medical  facilities, 
or  social  service  agencies.  If  they  are  not 
working,  and  are  living  alone  with  chil- 
dren, students  may  feel  isolated,  and  rarely 
have  chances  outside  our  classrooms  to 
develop  the  social  and  interpersonal  skills 
that  they  will  need  in  the  workplace  or  in 
higher  education.  Every  time  we  ask  stu- 
dents to  follow  class  rules,  to  debate  with 
each  other  according  to  certain  guidelines, 
or  open  up  discussion  about  social  science 
topics  like  psychology  or  sociology,  we  are 
helping  students  to  develop  this  very  im- 
portant skill. 

Questions  for  evaluating  students' 
knowledge/ skill  in  this  area  may  include: 

• Does  the  student  understand  the  differ- 
ence between  group  membership  and 
individual  identity?  Does  he/she  see 
how  those  two  roles  may  conflict? 

• Can  the  student  identify  causes  of  har- 
mony or  discord  among  groups  of 
people? 

• Does  the  student  understand  the  differ- 
ences that  may  surface  between  group 
interactions  and  one-to-one  interactions 
among  the  members  of  a group? 

• Does  the  student  see  how  cultures/soci- 
eties affect  the  behavior  of  different 
groups  (gender  groups,  religious 
groups,  etc.)? 

• Can  the  student  talk  comfortably  about 
the  difference  between  "nature"  and 
"nurture"? 

• Does  the  student  recognize  the  different 
ways  in  which  people  may  be  grouped 
within  a society,  and  some  of  the  factors 
that  determine  that  grouping? 
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Many  teachers  will  find  that  these  ques- 
tions can  be  pursued  in  written  work,  giv- 
ing students  the  opportunity  to  develop 
new  understandings  and  insights  more  ef- 
fectively than  they  can  in  discussion.  Good 
essay  topics  that  address  these  issues  can 
be  found  in  many  adult  literacy  and  GED 
prep  materials.  It  may  be  especially  useful 
to  have  students  write  about  and  then  dis- 
cuss these  issues  in  class,  thus  combining 
their  new  knowledge  about  the  ways  that 
groups  function  with  opportunities  to  put 
that  knowledge  into  practice. 

Skill  6:  The  ability  to  go  about  learning 
something  for  yourself,  and  the  skills  and 
confidence  to  be  a learner  all  your  life. 

In  many  ways,  this  last  skill  addresses 
all  the  others,  and  brings  me  back  to  the 
reason  I went  in  search  of  this  kind  of  as- 
sessment. Many  of  our  students  will  leave 
our  classes  before  achieving  their  goals, 
and  it  may  be  some  time  before  they  are 
able  to  return.  The  degree  to  which  we  are 
able  to  build  this  skill  determines,  more 
than  anything  else,  the  value  of  the  time 
they  spend  in  our  classrooms.  As  Kohl 
points  out,  we  "need  to  find  ways  to  help 
people  learn  for  themselves  so  that  they 
can  make  informed  decisions  on  major  is- 
sues that  affect  their  lives  rather  than  shift 
responsibility"  to  people  who  may  not 
have  the  knowledge  to  help,  or  who  may 
not  have  students'  best  interests  at  heart 
(1998, 151). 

Most  of  us  have  forgotten  the  bulk  of 
what  we  learned  in  school,  and  we  would 
be  unrealistic  if  we  expected  our  adult  stu- 
dents to  be  any  different.  As  important  as 
the  rules  of  grammar,  or  the  ability  to  do 
long  division,  is  the  ability  to  continue 
learning.  We  know  that  a GED  diploma 
does  not  guarantee  success,  though  it  is  a 
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significant  achievement.  As  teachers,  we 
need  to  see  our  work  not  as  getting  people 
over  hurdles,  but  as  equipping  them  to 
choose  their  own  hurdles,  and  clear  them 
under  their  own  power. 

Questions  for  this  skill  might  include: 

• Does  the  student  know  how  to  use  a dic- 
tionary? A thesaurus?  Has  he/ she  de- 
veloped the  habit  of  using  them? 

• Is  the  student  informed  about  current 
events  that  may  affect  his/her  life? 
Where  does  the  information  come  from? 
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informed  and  an  uninformed  opinion  on 
a given  issue? 


• Does  the  student  know  how  to  use  the 
Internet?  A library  card  catalogue? 


• Can  the  student  seek  out  knowledgeable 
people  in  situations  where  he /she  needs 
assistance?  How  does  he/ she  find  those 
people? 


• Is  the  student  aware  of  local,  state,  and 
national  resources  that  will  help  him/ 
her  to  achieve  further  goals? 


The  list  of  possible  questions  for  this 
skill  is  nearly  endless,  and  depends  greatly 
on  the  specific  needs,  abilities,  and  back- 
grounds of  individual  students.  Many  stu- 
dents will  struggle  in  their  attempts  to 
become  their  own  best  resources;  years  of 
difficulty,  and/or  school  failure,  often  un- 
dermine confidence,  and  diminish  stu- 
dents' willingness  to  take  risks. 


SUGGESTIONS  FOR  USE 

No  one  approach  to  this  framework  can  be 
considered  "best."  Rather,  the  benefit  of 
using  these  skills  as  a model  for  curricu- 
lum development,  course  structure,  and/ 
or  assessment  lies  in  each  individual 
instructor's  or  program's  decisions  about 
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what  will  benefit  students.  My  own  use  of 
these  skills  as  a way  of  thinking  about  stu- 
dent progress  has  proven  beneficial  both 
for  my  students  and  for  my  teaching,  be- 
cause it  has  given  us  a way  to  look  at  what 
should  be  considered  progress  toward 
educational  goals. 

The  most  significant  benefit  of  this  ap- 
proach in  my  own  classroom  has  been  the 
degree  of  holism  it  allows.  Thinking  about 
skills  in  terms  of  students,  rather  than  con- 
tent areas,  leads  me  to  make  better,  more 
integrated  choices  about  how  to  structure 
class  time,  and  how  to  combine  the  treat- 
ment of  various  subject  areas.  Addition- 
ally, when  I think  in  terms  of  students 
developing  skills,  rather  than  students  col- 
lecting knowledge,  it  becomes  easier  to  see 
where  and  when  progress  happens.  We 
have  not  abandoned  the  traditional  goals 
of  adult  education,  but  have  found  a 
richer,  more  nuanced  context  in  which  to 
place  them. 

Genuine  education  is  not  a matter  of  ac- 
cumulation, but  of  acculturation.  As  we  in- 
troduce concepts  and  academic  content, 
we  simultaneously  introduce  our  sense  of 
what  should  be  valued,  what  counts  as 


knowledge,  and  how  learning  should  take 
place.  Student  success  rests  as  much,  or 
more,  on  their  ability  to  follow  the  rules, 
implicit  and  explicit,  that  govern  institu- 
tions. For  adult  learners  who  wish  to  pur- 
sue higher  education,  job  retraining,  and 
employment,  learning  accepted  patterns  of 
behavior  and  classroom  practices  is  critical 
for  success. 

Looking  at  student  development  in  the 
terms  I have  outlined  allows  me  to  con- 
sider what  I need  to  do  to  help  each  indi- 
vidual become  an  educated  person  not 
necessarily  in  terms  of  a transcript  or  de- 
gree, but  as  it  affects  their  stance,  perspec- 
tive, and  way  of  approaching  the  world.  I 
would  suggest  that  mastery  of  the  skills 
presented  here  can  lead  to  increased  aca- 
demic independence,  self-confidence,  and 
reflectiveness,  the  qualities  we  say  we 
want  to  see  in  an  educated  person.  When 
we  pay  careful,  systematic  attention  to  the 
habits  of  mind  that  support  learning,  as 
Herbert  Kohl  urges  us  to  do,  we  move  a 
long  way  toward  making  our  students' 
time  in  our  classrooms  valuable,  practical, 
and  immediate. 
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Are  We  Practicing  What  We  Preach? 


Since  the  International  Language  In- 
stitute of  MA  (ILI)  opened  in  1984, 
the  modus  operandi  in  the  classroom 
has  been  integrating  activities.  Instead  of 
assigning  one  specific  activity  to  each  skill 
area,  we  take  one  activity  and  stretch  it 
(without  stretching  the  point  too  much)  to 
include  all  the  skill  areas.  For  example,  in- 
stead of  focusing  just  on  the  speaking/lis- 
tening skills  by  having  learners  discuss  a 
topic,  we  expand  the  activity  to  include  a 
reading  and  a writing  activity  connected  to 
that  topic.  This  way  of  approaching  lan- 
guage teaching  allows  learners  to  focus  on 
the  whole  picture  rather  than  a series  of 
disconnected  activities  practicing  the  dif- 
ferent skill  areas;  it  appears  that  our  learn- 
ers learn  better  this  way.  Watching  an 
entire  program  instead  of  channel  surfing 
with  a remote  allows  you  to  see  the  whole 
picture  in  context  rather  than  snippets. 
How  can  we  evaluate  (as  teachers  and  ad- 
ministrators) whether  we  are  actually  inte- 
grating activities  in  the  classroom  and  then 
reflect  on  its  benefits? 

It  is  important  that  we  understand  how 
integrating  activities  is  connected  to  the 
theoretical  way  we  look  at  language.  Being 
that  we  are  language  teachers,  what  better 
way  to  demonstrate  this  idea  than  with  a 
visual?  At  ILI,  we  use  the  "The  Wheel  of 
Concepts  of  Language"  to  show  how  the 
areas  of  language  are  connected  (see  Figure  1). 

The  wheel  is  used  because  of  the  idea  of 
moveable  parts.  In  the  center  of  the  wheel 
are  the  core  skill  areas  of  communication. 
(In  cultures  where  language  is  simply  an 
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oral  tradition,  there  are  two  main  areas  of 
communication:  listening  and  speaking. 
Since  English  has  both  an  oral  and  written 
tradition,  we  refer  to  the  four  skill  areas 
listed  above)  Outside  and  connected  to  the 
core  of  the  wheel  are  the  tools,  or  other 
concepts  of  language.  These  tools  play  a 
role  in  the  learners'  development  and 
strengthening  of  the  skill  areas,  and  his/ 
her  overall  ability  to  communicate.  As  one 
moves  out  from  the  center  of  the  wheel, 
each  section  moves  about  to  mix  and 
match  with  each  other,  and  can  do  so 
simultaneously.  For  example,  a speaking 
activity  may  work  on  pronunciation, 
grammar,  intonation,  and  culture  all  at  the 
same  time. 

At  ILI  the  Course  Progress  Report  we 
use  is  connected  to  the  Wheel  of  Concepts. 
The  Course  Progress  Report  (see  page  17) 
shows  how  our  classes  are  structured, 
documents  what  is  covered  in  the  class 
and  clarifies  how  classes  are  integrated. 

A checklist  is  part  of  the  Course  Progress 
Report  and  mirrors  the  Wheel  of  Concepts. 
At  the  end  of  each  class,  teachers  review 
the  course  progress  sheet  and  check  off 
which  areas  have  been  covered.  This 
checklist  allows  teachers  to  see  if  they  are 
integrating  activities,  which  skill  area 
needs  more  attention,  and  if  they  are 
spending  too  much  time  on  one  skill  area 
and  not  enough  time  on  the  others. 

COURSE  PROGRESS  REPORT 

The  Course  Progress  Report  includes 
lots  of  space  for  teachers7  notes.  Teachers 
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THE  WHEEL  OF  CONCEPTS 


FIGURE  1 

The  Wheel  of  Concepts 
was  designed  by  Alexis 
Johnson,  Executive 
Director  of  ILI.  It  can 
also  be  found  in  the 
Commonwealth  Literacy 
Corps  Volunteer  Tutor 
Training  Manual. 


decide  how  this  is  used  and  have  used  the 
space  to  jot  down  how  the  activity  worked 
(or  didn't)  along  with  ideas  for  future  ac- 
tivities, for  documenting  suggestions  from 
learners  that  came  up  during  feedback, 
and  as  a teachers'  journal  and  an  instru- 
ment for  teacher  reflection  at  the  end  of 
each  class. 

At  ILI  we  strive  to  have  students  speak- 
ing 60%-80%  of  the  time.  At  the  bottom  of 
the  Course  Progress  Report  is  the  question 
"Are  students  speaking  60-80%  of  the 
time?"  to  help  teachers  remember  that  our 
focus  is  getting  learners  to  speak. 

Also  at  the  bottom  of  each  Course 
Progress  Report  is  a series  of  reflective 
questions  for  teachers  to  ask  to  get  feed- 
back about  the  class.  What  activities/ skills 
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did  we  cover?  What  did  you  learn  from 
these  activities?  Would  you  have  changed? 
What  would  you  like  to  see  in  the  next 
class?  The  arrows  to  and  from  Evaluation/ 
Feedback  on  the  Wheel  of  Concepts  dem- 
onstrate the  connection  between  what  is 
going  on  in  class  and  the  importance  of 
feedback  and  evaluation  as  part  of  the 
structure  of  the  class.  Teachers  and  learn- 
ers alike  need  to  understand  why  these 
questions  are  being  asked  and  that  the  in- 
formation retrieved  from  these  questions  is 
being  used  to  drive  the  curriculum.  There- 
fore, feedback  from  learners  needs  to  be 
highlighted  in  a way  that  shows  learners 
that  we  are  listening.  This  is  an  important 
step  in  getting  learners  to  understand  that 
they  must  take  on  the  responsibility  for  . 
their  own  learning. 

We  rely  heavily  on  oral  feedback  at  the 
end  of  each  class.  Often  new  teachers  run 
out  of  time  and  when  asked  after  class  if 
they  have  done  feedback,  the  answer  is 
"No,  but  I'll  be  sure  to  do  it  tomorrow." 
Having  the  questions  on  the  Course 
Progress  Report  helps  teachers  remember 
that  these  questions  need  to  be  asked  con- 
sistently so  that  students  continue  to  drive 
the  curriculum.  We  also  include  a checkoff 
area  under  evaluation  on  the  Course  Pro- 
gress Report  to  ensure  that  it  has  been  done. 

We  spend  a lot  of  time  at  ILI  talking 
about  evaluation/ assessment  and  making 
sure  that  doing  feedback  at  the  end  of  the 
class  is  not  just  an  "add  on"  activity,  but 
something  the  students  see  consistently 
and  which  becomes  part  of  the  class.  The 
feedback  section  of  the  class  could  also  be 
an  integrated  activity:  in  pairs  learners  talk 
about  the  activities  and  their  feelings  about 
the  class/ their  progress.  This  is  written 
down  either  by  the  student  or  the  pair  and 
is  then  shared  with  other  students  in  the 
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class.  Getting  feedback  from  the  students 
in  this  way  helps  them  practice  their  listen- 
ing, speaking,  reading  and  writing  skills. 

Assessment  is  an  on-going  process  and 
we  know  that  we  all  need  constant  work 
on  how  we  handle  assessment  in  the  class. 
Reviewing  the  Wheel  of  Concepts  and  how 
we  use  our  Course  Progress  Reports  is  part 
of  ILI's  staff  development.  As  suggested 
by  teachers  in  a recent  workshop,  we  are 
also  going  to  include  the  Wheel  of  Con- 


cepts in  the  student  handbook  as  a way  of 
informing  learners  what  is  going  on  in  the 
class.  The  Wheel  of  Concepts  started  out  as 
being  just  for  teachers,  but  we  realize  that 
having  this  available  to  learners  will  en- 
able them  to  see  how /why  we  structure 
our  classes  the  way  we  do  and  the  impor- 
tance we  put  on  learner  feedback  and  inte- 
grating all  skill  areas.  It  will  also  keep  us 
honest  as  to  making  sure  that  we  practice 
what  we  preach. 
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This  is  Only  a Test 
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1 first  became  aware  of  assessment  in 
adult  education  during  the  initial  de- 
velopment of  the  BEST,  in  the  early 
'80s  (see  AIA,  Volume  10,  December,  1997, 
for  a current  consideration  of  the  BEST).  I 
had  been  working  at  a site  in  Rhode  Island 
that  had  participated  in  the  Center  for  Ap- 
plied Linguistics'  overall  development  of 
both  the  test  and  the  performance  levels  it 
measured,  but  never  clearly  saw  the  con- 
nection between  the  test  itself  and  educa- 
tional practice.  I began  to  understand  the 
notion  of  accountability  but  I could  never 
really  tell  how  a particular  score  on  the 
BEST  would  inform  my  teaching. 

I still  can't. 

I can,  of  course,  get  a sense  of  what  a 
learner's  ability  to  take  a test  such  as  the 
BEST  might  be  like  based  on  his/her 
scores  on  one.  I can  see  the  range  of  con- 
tent (life  skills,  math,  science,  reading  com- 
prehension) she  knows  and  the  kinds  of 
items  (essay,  multiple  choice,  true/ false, 
yes/ no  questions)  that  s/he  is  able  to  suc- 
cessfully complete  or  that  s/he  might  miss. 
I can  see  if  that  learner  has  'done  school,' 
and  to  what  extent,  but  for  the  most  part, 
when  working  with  adults  with  limited 
English  language  abilities  and  little  prior 
schooling,  most  standardized  tests  are  not 
teaching  me  very  much  about  learners' 
strengths  or  needs. 

I can  most  ably  assist  a learner  in  find- 
ing an  appropriate  class/level  placement 
and  meeting  her  educational  goals  by 
speaking  to  her,  by  looking  at  a writing 
sample,  by  learning  about  what  she  knows 
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about  in  the  world,  where  or  if  there's  been 
prior  schooling  and  in  which  language(s). 
This  information  gives  me  a sense  of  what 
a learner  knows  about  using  print  as  a re- 
source for  learning,  and  about  her/his 
sense  of  what  school  can  or  should  be. 
Learning  about  what  s/he  has  done  before 
coming  to  the  learning  center  gives  me  a 
sense  of  what  strengths  and  needs  s/he's 
bringing  to  the  learning  process. 

In  some  settings,  placement  tests  are  help- 
ful to  practitioners  in  appropriately  directing 
learners  into  the  levels  offered  within  par- 
ticular programs;  in  and  of  themselves,  how- 
ever, the  tests  have  limited  value.  Some 
programs  have  developed  their  own  tests, 
others  rely  on  standardized  tools.  For  many 
learners,  though,  any  test  is  daunting.  Learn- 
ers who  may  already  be  nervous  about  com- 
ing to  school  really  don't  need  to  have  an 
additional  element  of  risk  added  to  their  ini- 
tial encounter  with  the  learning  program. 

In  other  settings,  standardized  tests  are 
not  useful  to  educators  for  placement  or 
for  progress  assessment.  They're  either  too 
difficult  (so  that  beginning  level  learners 
'bottom  out,'  gaining  scores  that  hardly  in- 
dicate anything  about  their  abilities)  or  do 
not  measure  what  learning  has  actually  oc- 
curred within  a classroom.  Programs  feel 
caught  between  a rock  and  hard  place; 
they  have  to  show  some  pre-  and  post- 
learning  movement,  and  tend  to  minimize 
the  test  itself  to  learners  ("it's  just  to  help 
us  help  you  find  the  right  level/ get  a sense 
of  where  you  are  now/ show  that  you've 
been  here.'.-."). 
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Many  programs  have  taken  the  time  and 
energy  to  explore  how  testing  can  be  used 
more  appropriately,  or  how  alternatives 
can  be  developed.  Others  report  that  learn- 
ers who  do  show  gains  in  standardized  test 
scores  feel  encouraged  by  those  gains. 

But  many  practitioners  indicate  that 
tests  terrify  their  students,  or  waste  their 
time  or  that  they  can't  possibly  prepare 
students  to  take  so  many  different  tests.  ("I 
can't  send  one  out  to  a program  that  gives 
the  TABE,  another  to  a CASAS-based  pro- 
gram, and  yet  others  on  to  the  GED..."). 

I maintain  that  you  can,  but  that's  not 
the  point.  If  a learner  identifies  an  aca- 
demic goal  for  which  testing  will  be  an  im- 
portant and  ongoing  part  of  learning,  there 
are  strategies  and  approaches  to  prepare 
students  to  take  tests  and  to  pass  them. 
Looking  at  the  language  of  tests,  the  kinds 
of  reading  strategies  that  learners  can  use 
to  make  meaning  and  then  to  translate 
their  understanding  of  that  meaning  into 
essay,  multiple  choice,  true/ false,  or  yes/ 
no  answers  is  possible.  More  important, 
however,  is  the  question  of  what  it  is  the 
students  want  to  do. 

As  a state  literacy  resource  center  direc- 
tor, I not  only  work  with  and  think  about 
learners;  I'm  concerned,  too,  about  pro- 
gram progress  overall.  I need  to  look  at 
and  think  about  ways  to  measure  progress 
beyond  gains  made  by  individual  learners, 
and  to  see  how  groups  of  learners  progress 
over  time.  Looking  beyond  pedagogical 
concerns,  I also  need  to  be  able  to  demon- 
strate learners'  progress  to  those  who  fund 
programs  and  to  those  who  make  policy. 

As  a vehicle  of  accountability,  and  of  re- 
porting larger  spheres  of  need  (and  accom- 
plishment) to  those  who  do  not  work 
directly  with  learners  but  who  do  deter- 
mine funding  patterns  and  policy,  the  Na- 


tional Adult  Literacy  Survey  (NALS)  has 
most  recently  become  the  standard  against 
which  everything  we  do  is  measured  in 
some  way.  The  NALS  indicates  that  a siz- 
able percentage  of  the  population  of  this 
country  is  ill  prepared  to  engage  in  higher 
order  thinking  skills,  having  assessed 
those  skills  on  a pencil/ paper  test.  The 
underlying  assumption  here  is  that  higher 
order  thinking  can  (only)  be  demonstrated 
in  writing,  and  that  this  is  an  appropriate 
way  of  measuring  such  thinking.  The 
NALS  claims  that  it  assesses  real  life  skills, 
yet  it  measures  an  adult's  ability  to  use 
those  skills  through  a print-based  medium. 
This  is  neither  the  time  nor  the  place  for  a 
lengthy  analysis  of  the  NALS  or  its  meth- 
odology. Rather  it  is  a time  to  consider 
how  it  has  been  taken  up  as  the  largest 
umbrella  under  which  all  our  other  at- 
tempts at  using  test  scores  seem  to  fall. 

The  NALS,  understandably,  has  been 
used  to  establish  the  need  for  increased 
services  for  learners.  However,  as  a practi- 
tioner committed  to  not  perpetuating  defi- 
cit views  of  adult  learning  needs,  I find  it 
troublesome  to  contemplate  the  way  these 
needs  are  flaunted  through  a rendering  of 
NALS  scores.  Such  renderings  tend  to  em- 
phasize adults'  lack  of  abilities  with  lit- 
eracy in  order  to  play  up  the  need  for 
increased  access  to  educational  opportuni- 
ties for  adult  learners.  In  addition  to  flog- 
ging NALS  findings  within  their  states, 
programs  also  grapple  with  using  learner 
outcomes  that  are  based  on  a variety  of 
tests  — using  gains  in  test  scores  to  prove 
their  effectiveness  as  educational  provid- 
ers; using  low  scores  to  speak  to  the  need 
for  yet  more  services  to  be  provided.  None 
of  us  uses  the  NALS  to  measure  our  own 
learners'  actual  abilities,  however,  even  if 
we  accept  that  the  NALS  could  provide  us 
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with  useful  information  about  those  abili- 
ties. 

At  a recent  state-level  policy  meeting,  a 
suggestion  was  made  that  we  actively  seek 
correlations  between  the  NALS  and  other 
standards  (such  as  the  GED),  in  order  to 
set  more  precise  goals  around  learner  out- 
comes. Such  goal  setting  in  and  of  itself 
does  not  improve  learning,  though;  it 
merely  shifts  the  focus  (yet  again)  in  terms 
of  what  learners  and  educators  are  being 
asked  to  demonstrate. 

It  seems  I'm  really  addressing  two 
broad  issues:  actual  learning  and  account- 
ability and  the  endless  need  for  funding. 
Both  interest  me  in  different  ways.  How 
can  adminstrators  (and  teachers  and  learn- 
ers for  that  matter)  argue  intelligently  for  a 
meaningful  way  of  assessing  progress 
while  also  dancing  to  the  multiple  times 
played  by  the  funders  to  whom  they  are 
accountable?  How  do  we  translate  what 
we  know  about  learning  into  a form  that 
can  be  understood  readily  by  those  who 
fund  and  support  our  work? 

Again,  it  seems  that  speaking  about  test- 
ing and  accountability  is  speaking  about 
reporting,  translating  gams  that  students 
make  on  a particular  test  to  funders  or  in- 
terested others  to  whom  that  information 
is  important.  The  work  done  within  the 
last  decade  on  broadening  the  range  of 
what  counts  and  how  we  count  it  has  con- 
tributed significantly  to  the  field's  under- 
standing of  the  multiple  ways  in  which 
learning  can  be  measured.  We  still  seem 
woefully  incapable,  however  of  translating 
actual  learning  gains  into  something  that 
funders  know,  understand  and  appreciate 
as  indications  of  learning  having  taken 
place.  With  constant  turnover  in  the  field 
itself,  as  well,  we  often  need  ongoing  ways 
to  assist  ourselves,  as  practitioners,  to  learn 
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to  closely  observe  learners'  work  and  to 
translate  our  own  observations  into  better 
understandings  of  their  strengths  and 
abilities.  Such  understandings  can  inform 
our  attempts  to  improve  practice  to  meet 
learners'  needs  more  appropriately.  Adven- 
tures in  Assessment  is  one  way  through 
which  this  work  is  being  carried  out,  but 
more  work  on  a program-based  level  is  re- 
quired if  we  really  are  to  connect  assess- 
ment and  learning  and  move  away  from 
arbitrary  and  capricious  testing  and  more 
closely  into  assessment  that  informs  both 
learners  and  practitioners  in  meaningful 
ways. 

Heide  Wrigley  (personal  communica- 
tion, August,  1998)  suggests  that  we  need 
to  understand  not  only  what  works  in 
terms  of  teaching /learning  approaches 
and  progress,  but  also  "the  implications  ... 
for  test  development  or  assessment  both 
large  scale  — program  wide  — and  small 
scale,  in  the  classroom."  This  close  learning 
can  barely  be  gleaned  from  a barrage  of 
test  scores  - even  tests  that  look  at  'life 
skills.'  (Both  the  BEST  and  CASAS  have 
'authentic'  tasks,  if  you  consider  it  authen- 
tic to  talk  about  how  you  go  to  the  bank  or 
to  the  laundromat,  or  whatever  else  one 
does  without  explicit  talk  in  LI  or  L2.)  Yet, 
again  and  again  we  insist  on  using  stan- 
dardized tests  to  get  a sense  of  a group 
of  learners'  abilities  so  that  we  can  report 
these  abilities  to  funders.  Nonetheless,  I 
realize  the  significance  of  Heide's  sugges- 
tion that  standardized  tests  can  show  us 
progress  across  groups  more  clearly  than 
they  give  us  useful  information  for  teach- 
ing. Information  gleaned  from  test  scores 
can  help  us  to  see  how  groups  of  learners 
perform  over  periods  of  time.  They  give  us 
a sense  of  a body  of  learners  responding  to 
tests  of  a body  of  knowledge. 


How  do  we  trans- 
late what  we  know 
about  learning  into 
a form  that  can  be 
understood  readily 
by  those  who  fund 
and  support  our 
work? 
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We  need  to  know 
what  connecting 
assessment  to 
learning  looks  like, 
how  to  help  learn- 
ers see  their  own 
progress  through 
multiple  lenses 


Increased  assessment  — giving  more 
tests  — however,  can  not  improve  pro- 
grams or  learning  until  and  unless  those 
assessment  instruments  tell  teachers  about 
how  their  learners  learn  and  don't  learn. 
Heide  Wrigley's  article  in  this  issue  ad- 
dresses those  broader  program  and  policy 
concerns,  those  big  picture  pieces  that 
classroom  teachers  may  not  be  aware  of 
but  which  drive  their  work,  one  way  or  an- 
other, through  shifts  in  policy,  funding 
and  mandates. 

In  terms  of  local  programming  and  pro- 
fessional development,  Heide's  suggestion 
that  "we  can  either  go  to  a profile  system 
or  we  can  develop  a framework  that  com- 
bines some  deeper  classroom-based  assess- 
ments with  some  quicker  assessments  that 
are  done  across  classrooms  and  can  be 
compared  across  programs  (writing 
samples,  for  example,  or  participation  in 
an  interview  or  a discussion)"  makes  good 
sense.  As  Heide  suggests,  if  we  believe  that 
something  is  worth  teaching,  we  then  need 
to  know  if  students  have  learned  it.  How 
we  go  about  getting  that  information  is  a 
central  question. 

PROFESSIONAL  DEVELOPMENT 

As  indicated  above,  the  work  of  learn- 
ing how  to  understand  learner  progress 
needs  to  be  integrated  into  ongoing  profes- 
sional development.  We  need  to  know 
what  connecting  assessment  to  learning 
looks  like,  how  to  help  learners  see  their 
own  progress  through  multiple  lenses 
(longer  journal  entries,  fewer  mistakes  at 
the  blackboard,  recognition  of  positive 
changes  from  teachers  and  from  fellow 
learners).  Several  years  ago,  Francine 
Collignon  and  I undertook  an  action  re- 
search process  to  develop  one  form  of 
qualitative  assessment  of  a group  of  begin- 


ning level  ESOL  literacy  learners  at  the  In- 
ternational Institute  of  Rhode  Island.  The 
resulting  grid  was  a useful  vehicle  for  us  to 
review  learner  progress  on  a monthly  level 
and  to  show  our  funder  'evidence'  of  that 
progress.  The  process  of  reviewing  notes 
on  classes,  learners'  writing,  interaction  in 
class  and  general  activity  informed  those 
grids  and  helped  us  shift  our  teaching 
practice  accordingly.  We  never  found  a 
standardized  test  to  help  us  get  this  infor- 
mation, nor  were  we  obliged  to  submit 
scores  throughout  the  course  of  that  par- 
ticular three-year  grant.  The  work  of  docu- 
menting classes,  reviewing  those  notes  and 
discussing  learner  gains  was  time  consum- 
ing. We  had  a relatively  small  number  of 
learners,  and  time  for  preparation  and 
analysis  of  our  work.  Most  programs  do 
not  have  these  necessary  luxuries. 

There  are  'down  and  dirty'  ways  of 
looking  at  learner  progress.  In  addition  to 
deciding  what  it  is  we  want  to  assess 
(progress  with  writing?  pronunciation? 
mastery  of  certain  grammatical  forms?), 
there  are  other  factors  that  contribute  to 
learning:  class  size,  previous  education, 
situational  and  dispositional  barriers,  gen- 
der, age  and  myriad  other  realities  of 
adults'  lives.  Classroom  teachers  know 
that  these  factors  are  all  as  important  to 
learning  as  the  quality  of  teaching  itself. 
We  still  see,  though,  that  a better  test 
doesn't  give  us  a better  program.  So  where 
are  we  now? 

It  seems  that  practitioners  and  adminis- 
trators are  still  grappling  with  the  issues 
I've  raised  above.  Nancy  Fritz,  an  ESOL 
teacher  at  the  Genesis  Center  in  Provi- ' 
dence,  is  working  on  a mini-grant  to  study 
existing  assessment  protocols,  in  the  hopes 
of  adapting  one  to  use  with  her  students 
and  to  share  with  colleagues  as  the  Center 
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considers  revising  its  curriculum.  Nancy 
speaks  to  her  concerns  as  a classroom 
teacher  and  as  a member  of  a particular 
educational  community: 

During  the  1997-98  academic  year,  I 
taught  an  intermediate  level  class  ofESOL 
adult  learners  in  a program  that  provides 
ESOL  instruction  at  four  different  levels  for 
five  hours  a day,  five  days  a week.  During  the 
year,  I came  to  realize  some  of  the  problems 
inherent  in  trying  to  assess  our  learners' 
progress.  This  experience  has  led  me  to  try  to 
find  or  develop  some  instruments  that  will  be 
helpful  to  me,  other  teachers  and  to  our  stu- 
dents in  measuring  growth  in  English.  I'm 
doing  this  with  the  support  of  a mini-grant 
from  the  Literacy  Resour ces/R.I.  - the  state's 
literacy  resource  center. 

In  our  ESOL  classes  we  currently  use  the 
written  form  of  the  BEST  test  which  is  admin- 
istered at  the  beginning  of  the  year  and  again 
at  the  end  in  June.  The  problem  that  I see  with 
the  BEST  test  is  that  it  is  very  difficult  for 
many  of  our  beginning  level  students.  They 
have  to  guess  at  many  of  the  answers  and  if 
they  happen  to  guess  better  in  September  than 
they  do  in  June,  their  scores  actually  go  down. 

I saw  this  happen  to  a few  students  who  had 
shown  a great  deal  of  improvement  in  reading, 
writing,  and  speaking  English  during  the  year. 
The  BEST  test  did  not  reflect  the  progress. 

In  addition  to  the  BEST  test,  we  kept  an  on- 
going portfolio  assessment  for  each  student.  I 
think  the  idea  of  portfolio  assessment  is  great, 
but  the  lack  of  time  and  the  necessity  to  teach 
a large  number  of  students  made  it  impossible 
for  me  to  do  a good  job.  In  putting  together  the 
portfolios,  I also  felt  that  we  lacked  any  means 
of  measuring  the  progress  that  had  been  made. 
Progress  was  evident  if  one  took  the  time  to 
read  the  material  in  the  portfolio,  but  for  a 
quick  review  this  is  difficult. 


I am  looking  for  assessment  instruments 
that  will  yield  a quantifiable  result.  Many 
of  my  students  asked  me  during  the  year  for 
'grades'  or  'report  cards'  like  their  children 
receive  in  school.  I am  not  interested  in  such 
a cut-and-dried  approach  to  documenting 
progress,  but  I would  like  to  be  able  to  show 
measurable  results  to  students.  In  spite  of  the 
fact  that  they  knew  their  English  was  getting 
better,  they  could  provide  numerous  examples 
of  things  they  could  do  now  that  they  couldn't 
do  before,  and  they  could  see  their  progress  in 
their  portfolios,  they  wanted  this  tangible  'mea- 
sure' put  un  their  English  progress.  1 would 
like  to  be  able  to  take  two  pieces  of  writing  done 
at  intervals  of  several  months  and  compare 
them  according  to  criteria  that  are  realistic  for 
our  learners  and  that  also  serve  as  a guide  to 
good  writing  for  them.  The  writing  samples 
would  receive  a score  based  on  how  well  they 
measure  up  to  the  criteria.  In  think  this  proce- 
dure could  also  be  done  with  samples  of  spoken 
English.  1 think  that  this  type  of  assessment 
would  satisfy  our  desire  for  holistic  assessment 
while  at  the  same  time  provide  a quantifiable 
result  that  would  please  the  students,  give 
them  a sense  of  accomplishment,  and  make  a 
quick  review  possible  for  teachers.  I'm  hopeful 
that  I can  find  such  instruments  to  use  during 
the  coming  year. 

Nancy  worries  that  "wanting  some 
quantifiable  assessment  isn't  too  contrary 
to  the  ideas  in  Adventures  in  Assessment. 

I'm  not  really  obsessed  with  numbers  and 
I understand  that  there  are  lots  of  draw- 
backs. I just  think  it  would  be  helpful  to 
have."  This  concern  indicates  to  me  part  of 
what's  been  ingrained  in  adult  education 
people  for  many,  many  years.  The  need  for 
numbers  exists  in  constant  tension  with  the 
need  for  actual  information  with  which 
learners  can  see  their  own  progress,  teach- 
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Testing  might 
give  us  some 
information  about 
how  well  they've 
learned  whatever 
content/skills  we 
hope  to  have 
taught,  but  we 
need  to  be  mindful 
of  what  is  it 
learners  are  hoping 
to  gain,  what  we 
want  these  tests  to 
do,  and  to  tell  us. 


ers  and  learners  can  talk  about  that 
progress  and  most  importantly  actual 
learning  can  be  seen  and  increased.  What 
we  need  seems  light  years  away  from  what 
we're  finding  on  the  TABE,  on  the  BEST, 
through  CASAS  and  even  the  GED. 

Nancy  will  most  likely  develop  an 
adapted  tool  from  the  many  instruments 
she's  examining  and  will  share  her  find- 
ings with  her  colleagues  at  the  Genesis 
Center  and  with  others  in  a workshop  later 
this  fall.  I wonder,  though,  how  far  this 
learning  will  be  able  to  assist  us  as  a state 
(and  by  extension,  I hope,  as  a field)  in 
finding  a more  satisfactory  way  of  report- 
ing learners'  progress.  Meeting  with  shar- 
ing/ discussion  groups,  posting  findings 
on  Literacy  Resources/RI's  website,  and 
informally  communicating  with  colleagues . 
are  all  ways  to  consider  getting  the  word 
out  about  Nancy's  work  and  about  other 
possibilities  around  assessment.  We  seem 
to  endlessly  consider,  though,  what  useful 
assessment  does  and  looks  like,  with  little 
hope  of  implementing  'official'  assessment 
methods  that  funders  can  accept  (see,  for 
example,  NCLE's  posting  on  research  into 
assessment,  9/98). 

One  hopeful  movement  across  this  state 
and  across  the  country  is  the  evolution  of 
the  standards  designed  through  Equipped 
for  the  Future,  an  undertaking  of  the  Na- 
tional Institute  for  Literacy  (NIFL).  Sondra 
Stein,  NIFL  Senior  Research  Associate  and 
Director,  Equipped  for  the  Future,  relates 
that  Equipped  for  the  Future  lessons  "start 
from  what  adults  need  to  know  and  be 
able  to  do  in  REAL  LIFE,  integrate  skills 
instruction  into  work  on  meaningful 
projects,  and  assess  learning  results  by 
looking  at  the  application  of  skills  in  the 
context  of  carrying  out  real  life  tasks."  Dr. 
Stein  acknowledges  that  there  is  "a  ways 


to  go  before  this  translates  into  a frame- 
work for  accountability"  and  proactively 
invites  the  field  to  contribute  to  and  par- 
ticipate into  this  next  step  of  the  project. 

Heide  takes  up  the  difficulties  inherent 
in  assessing  even  skills  and  tasks  that 
learners  themselves  identify  as  being  im- 
portant. While  not  wanting  to  lose  sight  of 
those  difficultires,  I am  also  hopeful  about 
the  learning  that  may  occur,  at  least  within 
programs  and  classrooms,  as  a result  of 
the  work  on  the  EFF  standards  as  part  of 
a broader  ongoing  series  of  professional 
development  events  and  activities.  These 
events  and  activities  can  include  practitio- 
ner inquiry,  development  of  in-house  as- 
sessment tools,  attention  to  active  listening 
and  observation,  work  with  learners  in 
naming  goals  and  learning  about  how 
progress  occurs,  and  investigations  into 
learning  styles.  The  possibilities  are  many. 

In  the  long  run,  though,  the  same  thorny 
issues  remain.  No  matter  how  well  teach- 
ers learn  how  to  meet  their  learners'  needs 
and  work  from  their  strengths,  it  seems 
there  will  always  be  someone  external  to 
the  learning  process  who  will  want  to 
know  how  it  is  we  know  that  learners  have 
learned  something  and  what  it  is  that  they 
have  learned.  Testing  might  give  us  some 
information  about  how  well  they've 
learned  whatever  content/ skills  we  hope 
to  have  taught,  but  we  need  to  be  mindful 
of  what  is  it  learners  are  hoping  to  gain, 
what  we  want  these  tests  to  do,  and  to  tell 
us.  An  increase  in  testing,  or  a shift  in  the 
focus  of  testing  will  not  in  and  of  itself  im- 
prove learning.  It  might  improve  report- 
ing, but  will  anyone  learn  more  because 
their  teachers  are  using  new  or  different 
vehicles  of  testing  and  reporting?  Keeping 
these  questions  in  the  foreground  might 
help  program  workers  as  they  continually 
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work  on  the  dilemma  of  measuring  learn- 
ing in  ways  that  keep  everyone  informed, 
funded,  and  most  importantly,  learning. 

WEB  RESOURCES  OF  INTEREST 

Handouts  for  "How  Do  We  Measure 
Progress  in  Adult  ESL?"  From  an  online 
discussion  about  adult  education  through 
the  National  Clearinghouse  for  ESL  Lit- 
eracy Education  (NCLE).  http:// 
www.cal.org/discuss/adultesl/eslprog/ 
handoutl.htm 

Let's  Get  Started:  An  initial  assessment 
pack  for  adult  literacy  programs.  An 
on-line  booklet  of  information  intended  to 
help  a tutor  get  started  with  a new  learner 
from  Manitoba  Education  and  Training. 
http://www.nald.ca/clr/getstarted/ 
cover.htm 

Portfolios  in  Practice.  Andrea  Leis  with 
Rosemary  Rognvaldson.  On-line  at:  http: 
//www.nald.ca/dr/portfolio/cover.htm 
Hard  copies,  complete  with  graphics  not 


available  on-line,  can  be  purchased  for 
$5.60  each  from  Conestoga  College.  Send  a 
check  made  out  to  "Conestoga  College"  to: 
Bob  Mclver,  Conestoga  College,  435  King 
Street  North  Waterloo,  Ontario  CANADA 
N2J  2Z5  Phone:  (519)  885-300  x241,  FAX: 
(519)  747-1195  email: 
bmciver@conestogac.on.ca  Manuals  will 
be  sent  COLLECT  via  Purolator. 

http://www.cal.org/ncle/agenda.  Research 
agenda  for  Adult  ESL,  prepared  by  NCLE 
in  collaboration  with  the  National  Center 
for  the  Study  of  Adult  Learning  and  Lit- 
eracy (NCSALL)  with  support  from 
Teachers  of  English  to  Speakers  of  Other 
Languages,  Inc.  (TESOL).  The  paper  identi- 
fies issues  recommended  for  further 
research  and  development  in  the  field 
of  adult  ESL  education. 

http://www.brown.edu/departments/ 

swearer_center/literary_resources/ 

Janet  Isserlis'  website. 
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Reflections  at  the  End  of  an  ESL  Day 


A day  teaching  English  as  a second 
language  to  adults  is  a day  filled 
with  constantly  changing  situa- 
tions. I'm  sure  that  we've  all  been  in  the 
classroom  when  an  idea  that  was  to  be 
used  as  a 10  minute  warm  up,  fill  in  or  re- 
view ends  up  as  an  hour  long  activity,  just 
as  an  involved  series  of  tasks  sometimes 
has  to  be  cut  short  or  even  eliminated. 
Teaching  instincts  and  our  individual  flex- 
ibility allow  lots  of  changes  to  take  place 
within  the  classroom,  in  a way  that  feels 
normal  and  natural.  This  in  turn  creates  a 
relaxed  and  spontaneous  atmosphere  for 
learning  to  take  place.  Whether  your  lesson 
plans  are  formal  and  written  down  or  sim- 
ply stored  in  your  head,  and  whether  they 
are  carried  out  in  tact,  or  changed,  it  is  al- 
ways good  planning  to  run  a checklist  at 
the  end  of  the  hour,  lesson  or  day.  This  is 
in  effect  a self-evaluation,  a reflection  of 
what  happened  in  your  class  and  with 
your  teaching.  The  checklist  might  look 
something  like  the  following: 


by 

Joanna  Scott 


Westford,  MA 


End  of  Class  Reflection  List 

a)  was  the  aim/ goal  of  the  lesson/ activity 
achieved? 

b)  were  the  students  happy,  working  and 
interested? 

c)  was  I well  prepared? 

d)  with  hindsight,  what  would  I do  differ- 
ently? 

e)  was  I checking  students'  understanding 
throughout  the  activity? 

f)  did  I use  good  correction  techniques? 

g)  what  techniques  did  I use  to  elicit  an- 
swers, conversation  from  the  students 
(also  known  as  nomination  techniques)? 


ERIC 


TNTURES  IN  ASSESSMENT 


Keeping  this  list  at  the  front  of  your  lesson 
planning  book  or  at  the  back  of  your  mind 
really  serves  as  a good  framework  or  re- 
minder. It  helps  me  keep  an  ongoing  con- 
versation with  myself  and  also  my  class 
that  feels  somewhat  organized,  or  struc- 
tured in  nature.  It  makes  the  next  day's  les- 
son planning  more  logical,  more  coherent 
to  the  previous  day's  because  I'm  able  to 
draw  from  my  analysis  of  my  teaching,  the 
students'  behavior,  learning,  and  under- 
standing and  build  on  what  I have  learned. 

I focus  on  the  last  three  points  of  the 
reflection  list.  They  are,  first  of  all,  very 
much  related;  they  work  together  in  the 
process  of  this  self  evaluation.  These  three, 
questions  of  checking  understanding, 
using  good  correction  techniques,  and  elic- 
iting responses  often  help  me  to  build  rela- 
tionships with  my  students  and  to  plan 
ways  to  encourage  the  students'  participa- 
tion in  class.  The  way  we  use  these  meth- 
ods affects  how  we  teach,  our  interaction 
with  students,  and  how  we  plan  activities 
and  whole  lessons.  Ultimately  they  teach 
us  how  to  do  our  jobs. 

It  is  also  important  to  remember  that 
students  need  to  understand  what  we  are 
doing  and  to  know  what  is  expected  of 
them.  Our  behavior,  the  techniques,  and 
the  methods  we  use  must  stimulate  stu- 
dents, to  provide  a level  of  comfort  and 
challenge  at  the  same  time.  Students  will 
become  familiar  with  our  methods,  will 
pick  up  on  what  we  are  doing,  and  know 
what  is  expected  of  them  when  we  are  con- 
sistent with  how  we  question,  correct,  or 
elicit.  This  is  how  we  build  relationships 
with  our  students. 

One  program  I have  been  teaching  in 
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recently  — a year-round,  five-day-a-week 
program  — also  has  given  me  the  luxury 
of  time  to  try  to  learn  from  these  ideas  and 
to  build  these  relationships.  Too  often 
though  we  teach  in  programs  where  the 
lesson  time  is  short  and  infrequent,  mak- 
ing the  consistency  of  our  methods  of 
checking  understanding,  correcting  and 
eliciting  all  the  more  important,  being  sure 
that  we  communicate  our  expectations 
clearly,  and  that  we  get  feedback  and  par- 
ticipation from  our  students. 

vncvivinv  vnuKauiHniimv 

It  is  important  to  be  aware,  using  tech- 
niques I will  discuss,  of  students'  real  un- 
derstanding of  a grammar  point,  new 
vocabulary,  or  simply  the  conversation  go- 
ing on  in  the  classroom.  It  becomes  much 
easier  and  more  efficient  to  make  a deci- 
sion as  to  whether  to  continue,  or  whether 
to  adapt  or  review  the  ongoing  activity, 
lesson,  or  material  being  taught  or  being 
reinforced. 

Checking  understanding  is  one  of  the 
important  parts  of  the  reflection  checklist 
that  needs  to  happen  throughout  the  les- 
son as  well  as  at  the  end.  The  information 
you  receive  as  a teacher  affects  how  you 
teach  in  the  future  and  what  the  next  steps 
need  to  be.  Although  most  of  us  do  this 
anyway,  it  is  important  to  revisit  and  re- 
view how  and  when  we  do  so. 

Some  of  the  times  when  it  is  necessary 
to  check  understanding  might  be  when: 

1)  you  are  presenting  new  language  items. 

2)  a problem  has  arisen  in  the  course  of  a 

lesson,  or  when  you  realize  a student 

has  not  grasped  something. 

3)  you  are  correcting  student  errors. 

4)  you  are  giving  instructions. 

ERK * 
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HOW  DO  WE  DO  THIS? 

The  most  common  way  is  through  ques- 
tioning students  on  an  ongoing  basis. 
However,  the  question  "Do  you  under- 
stand?" does  not  usually  work  for  several 
reasons: 

1)  the  student  may  actually  think  s/he 
understands,  but  doesn't. 

2)  s/he  may  not  want  to  admit  not  under- 
standing, perhaps  for  fear  of  losing  face, 
taking  time,  or  just  fear  of  talking. 

3)  a student  may  not  be  clear  what  part  of 
the  lesson  s/he  is  supposed  to  under- 
stand; the  question  itself  is  not  clear. 

Here  is  an  example  of  using  productive 
questioning  to  check  understanding: 

During  a lesson  about  English  idioms 
and  expressions,  paraphrase  or  simplify 
the  expression  you  are  teaching  to  see  if 
students  have  understood  it. 

For  instance,  if  you  are  teaching  the  ex- 
pression 'it  isn't  necessary,'  first  change  it 
to  'don't  have  to'. 

Having  simplified  the  expression,  use 
that  simplification  to  create  an  interactive 
dialogue  between  you  and  the  class  or  stu- 
dent. Then  pose  the  question  to  the  class  or 
student: 

Teacher  "You  don't  have  to  bring  a gift 
to  a dinner  party,  do  you?" 

If  the  student  has  grasped  what  you  are 
teaching,  the  answer  will  show  the  under- 
standing. 

Student:  "No,  you  don't  have  to,  the 
host  does  not  expect  it;  it  isn't  necessary." 

This  tells  you  that  the  expression  has 
truly  been  understood  and  that  the  student 
knows  how  to  use  it  in  conversation.  If  the 
response  is  not  similar  to  this  example,  you 
now  know  to  re-teach  the  lesson  in  another 
way. 

Another  way  to  use  questioning  effec- 
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tively  to  check  understanding,  is  to  use  a 
statement,  followed  by  questions.  For  in- 
stance, if  you  are  teaching  a new  vocabu- 
lary word  like  'hesitated' , the  dialogue 
might  be: 

Statement  to  student:  "She  hesitated  before 
answering  the  question." 

Question  to  student:  "Did  she  answer  the 
question?" 

Student  response:  "Yes." 

Question:  "Did  she  answer  it  immediately?" 
Student  response:  "No." 

Question:  "Did  she  wait  for  a moment  be- 
fore answering." 

Student  response:  "Yes,  she  hesitated." 

Again,  an  inappropriate  answer  will 
alert  you  to  look  back  at  the  way  you 
taught  the  lesson  and  adapt  that  method 
and  then  use  questioning  to  check  under- 
standing again. 

Other  ways  of  checking  understanding 
can  include: 

1)  Explanations  given  by  the  students 
themselves,  e.g.  a student  explains  the 
word  'mortgage'  by  saying  "you  buy  a 
house  and  write  a check  to  the  bank  ev- 
ery month."  True,  but  a limited  state- 
ment. The  teacher  must  then  follow  with 
questions  to  check  that  the  other  stu- 
dents also  understand,  for  instance:  " 
Who  actually  owns  the  house  until  the 
mortgage  payments  (monthly  checks) 
are  all  paid?"  or  "How  long  do  you  have 
to  send  the  checks?  or  " Are  you  buying 
your  home  if  you  pay  rent?" 

2)  Using  contrasts  / oppositions  e.g  when 
learning  adjectives,  "What  is  the  oppo- 
site of  'tali',  or  'noisy'. . . ('acting  out'  ad- 
jectives helps  a lot  here  too,  particularly 
when  the  students  are  doing  it  too,  per- 
haps turning  the  lesson  onto  a guessing 
game,  charades....)  or  providing  students 
with  contrasting  statements  to  define: 


1.  "I  really  don't  like  pizza" 

2.  "I  don't  really  like  pizza" 

If  the  statements  are  understood,  the 
students'  definitions  should  make  it  appar- 
ent that  the  person  in  statement  1 feels 
much  more  strongly  about  not  liking  pizza 
than  the  person  in  2. 

3.  Prompting:  The  teacher  can  provide 
prompt  words  to  check  that  students 
can  make  meaning  of  a structure  as  in 
the  following: 

Student:  I used  to  smoke  a lot 

Teacher:  But ? 

Student:  But  I don't  now. 

This  tells  us  that  'used  to  ' is  understood 
to  be  in  the  past. 

4.  Translation  is  also  an  important  tool  in  a 
monolingual  situation.  There  are  inher- 
ent dangers  though:  the  first  language 
can  be  too  easily  become  the  preferred 
method  of  communication,  and  there 
aren't  always  translation  equivalents. 

5.  Using  timelines,  followed  by  questions, 
to  check  understanding  of  verb  tenses. 
Students  indicate  on  the  line  what  state- 
ment suggests: 

Diagram  on  board: 

now 


x_ 

I arrived  - the  train  left 

Question:  "Did  the  train  leave  before  or 
after  I arrived?" 

"Did  I arrive  before  or  after  the  train  left?" 
Statement: " When  I arrived,  the  train  left." 

now 

x 
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When  I arrived  the  train  had  left  the 

train  left  I arrived 

Use  the  same  line  of  questioning  for  each 
tense  you  teach. 

Be  as  thorough  as  possible  without  los- 
ing the  students'  interest.  Checking  can  be 
oral,  but  using  the  blackboard  provides  a 
visual  record  of  meaning. 

A high  level  of  understanding  and 
learning  depends  to  a large  extent  on  the 
material  being  presented  at  the  right  level, 
and  being  well  presented  with  activities 
designed  to  capture  and  keep  a student's 
attention.  It  is  also  dependent  on  some 
other  factors  that  were  mentioned  in  the 
reflection  list,  namely  correction  and  elicit- 
ing techniques. 

CORRECTION  TECHNIQUES 

Given  that  the  best  learning  usually  takes 
place  when  students  are  involved  and 
comfortable  enough  to  speak  out,  when 
they  are  not  afraid  to  make  a mistake,  an 
important  goal  is  to  decrease  the  teacher 
talking  time  (ITT)  and  increase  student 
talking  time  (STT). 

To  do  this,  the  ways  we  correct  students 
become  critical.  We  need  to  create  situa- 
tions, opportunities,  and  time  for  students 
not  only  to  want  to  speak  out  but  to  want 
to  correct  themselves  as  often  as  possible 
and  feel  proud  of  doing  so.  There  are  strate- 
gies we  can  use  to  deal  with  errors  in  all 
respects  including  those  that  come  up  as  a 
result  of  checking  understanding  and  from 
what  we  hear  when  we  use  different  nomi- 
nation techniques. 

The  principle  is  simple.  When  a student 
makes  an  error  in  grammar  or  pronuncia- 
tion and  you  feel  it  is  necessary  or  appro- 
priate to  correct  it  right  then,  consider  the 
following: 


Student:  "I  can  to  swim." 

Teacher:  "I  can...?"  using  questioning 
expression. . .encouraging  hand  signals. . . 
and  wait  for  an  alternate  response. 

This  can  be  repeated  more  than  once, 
or  maybe  opened  up  to  the  class  to  answer, 
having  the  original  student  repeat  or  just 
continue  on,  as  you  deem  appropriate. 
Basically,  the  fewer  words  the  teacher 
speaks,  the  better;  simply  repeat  the  sen- 
tence up  to  the  point  of  error  and  wait. 

If  the  teacher  simply  provides  the  an- 
swer, not  only  does  less  learning  take  place 
but  you  disallow  the  student's  pleasure 
and  pride  of  stating  the  correct  answer.  I 
believe  that  by  correcting  effectively,  we 
can  increase  understanding  dramatically. 

There  is,  of  course,  always  the  question 
of  whether  to  correct,  and  if  so,  whether  to 
do  so  at  the  point  of  error,  or  later.  Errors 
that  should  probably  be  corrected  are  those 
that  interfere  severely  with  communication 
between  students  or  those  that  would 
make  it  difficult  for  students  to  understand 
the  next  phase  of  a lesson.  If  you  realize, 
for  instance,  that  a student  has  miscon- 
strued a new  vocabulary  word  such  as 
'consistent/  thinking  it  means  'disagree- 
able/ then  your  plan  to  use  it  in  some  free 
discussion  will  not  be  productive.  It  needs 
correction. 

The  next  question  is  whether  the  whole 
class  needs  help  or  just  one  person,  and 
then  the  best  way  to  carry  out  the  correc- 
tion. To  alleviate  embarrassment  for  an 
individual,  initially  praise  (it  must  have 
validity)  the  student  for  their  good  pro- 
nunciation, the  good  sentence  they  said 
prior  to  the  error,  or  what  is  appropriate, 
and  approach  the  error  second.  Give  a sen- 
tence using  'consistent'  correctly,  and  ask 
how  it  sounded,  or  throw  the  question  of 
meaning  out  to  the  class  to  discuss,  while 
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and  feel  motivated. 
Should  the  class 
seem  reticent  in 
these  situations 
however,  the 
material  may  be 
too  hard  and  need 
to  be  modified. 


thanking  the  original  student  for  their  in- 
put. In  other  words,  like  in  most  situations 
related  to  work  in  the  classroom,  empha- 
size the  positive  and  don't  dwell  on  the 
negative.  If  students  are  reading,  I try  to 
interject  the  right  pronunciation  quietly 
and  let  them  repeat  it  essentially  without 
interruption.  I find  that  other  students  do 
it  anyway  and  in  general  it  is  non-threaten- 
ing in  a class  where  the  atmosphere  has 
been  encouraged  to  be  one  of  supporting, 
helping  one  another.  The  less  anxiety  the 
better! 

NOMINATION  TECHNIQUES 

Just  as  checking  understanding  and  effec- 
tive correction  techniques  are  skills  to  be 
learned,  using  various  ways  of  nomination 
also  require  practice,  and  planning  at  first. 
Some  techniques  to  try  are: 

1.  Eye  contact.  This  uses  no  language  and 
so  keeps  distraction  to  a minimum. 

2.  Random.  Keeps  everyone  awake  because 
students  cannot  predict  when  they  will 
be  called  on. 

3.  In  order.  Fair,  predictable  but  can  cause 
students  to  lose  focus  on  work  going  on 
because  they  are  so  busy  anticipating 
what  they  might  have  to  answer. 

4.  Free,  volunteer.  Particularly  good  at  point 
of  lesson  or  free  stage  when  real  life 
situations  are  being  acted  out. 

By  eye  contact  I mean  to  catch  a student's 
attention  and  silently  communicate  that 
you  would  like  for  them  to  respond.  It 
should  be  non-threatening  — no  names  are 
called  out  — but  if  it  is  clear  that  the  stu- 
dent can't  or  won't  answer,  you  can  con- 
tinue to  another  student  or  choose  another 
technique  to  use.  I like  to  use  this  a lot  be- 
cause it  is  quiet  and  my  voice  is  not  inter- 
rupting or  distracting  the  ongoing  work.  It 


also  gives  choice  to  the  students  because 
they  can  avoid  the  teacher's  eye  if  they 
want  to.  Instinct  will  tell  you  when  to 
change,  whether  you  feel  it  necessary  to 
have  someone  participate  or  not. 

Random  nomination  can  also  work  well 
when  you  need  to  keep  a particular  indi- 
vidual or  group  alert.  In  this  method,  you 
would  call  specific  names  of  those  you 
want  to  be  involved.  If  you  have  students 
who  are  particularly  shy  or  anxious,  and 
you  can  tell  them  ahead  that  they  just  need 
to  listen,  that  you  won't  call  on  them  until 
they  are  ready.  Their  relief  will  allow  them 
to  learn  much  better! 

Try  to  observe  reactions.  Are  you  keep- 
ing students  alert  without  being  nervous 
or  anxious?  Are  they  drifting,  falling 
asleep?  Is  the  pace  of  the  class  being 
slowed  down  or  becoming  too  predictable? 
Are  students  waiting,  hoping  to  partici- 
pate? Be  aware  that  nominating  gives  you, 
as  teacher,  control,  e.g.  "Maria,  tell  us. .." 

I find  that  it  is  good  at  times  to  have  the 
students  choose  the  next  person,  nominee. 
Think  about  how  much  control  you  want 
to  be  comfortable.  Think  too  about  what 
makes  your  class  more  comfortable.  The 
random  method  can  be  very  scary  to  a shy, 
nervous  learner  but  success  for  that  indi- 
vidual will  build  tremendous  self  confi- 
dence and  serve  too  to  encourage  his/her 
classmates.  We  have  a responsibility  to 
help  students  take  a risk,  to  speak  out  and 
feel  motivated.  Should  the  class  seem  reti- 
cent in  these  situations  however,  the  mate- 
rial may  be  too  hard  and  need  to  be 
modified.  This  is  also  an  opportunity  to 
check  understanding  during  the  activity, 
to  support  and  encourage. 

When  we  elicit  answers  in  order,  there 
may  be  two  opposing  reactions  within 
your  group  of  students.  Some  will  enjoy 
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knowing  when  they  will  be  called  on,  and 
relax  meantime.  Others  won't  be  able  to  re- 
lax because  they  will  focus  all  their  atten- 
tion on  their  turn,  and  may  lose  out  on 
learning. 

A large  class  lends  itself  to  free,  open 
nomination,  which  can  be  freeing  for  all, 
releasing  pressures  sometimes  felt  in 
smaller  classes.  I have  used  this  method 
with  success  in  higher  level  classes,  where 
there  is  often  more  confidence.  Open  the 
classroom  floor  to  students,  to  allow  those 
willing  to  answer  freely.  This  often  creates 
a noisy,  stimulating  atmosphere  of  ideas 
and  thereby  encourages  those  more  reti- 
cent to  participate.  The  teacher's  role  is 
then  to  be  a moderator. 

In  general,  we  will  select  the  methods 
best  suited  to  our  style  of  teaching,  and  to 
the  make  up  of  a class,  but  it  can  help  us  to 
expand  students'  learning  if  we  are  willing 
to  try  new  ways  of  nomination.  The  goal  is 
always  to: 


1.  elicit  answers. 

2.  encourage  participation. 

3.  check  understanding. 

4.  correct  errors. 

5.  revise  work. 

6.  give  or  keep  control. 

It  is  the  role  of  a teacher  to  help  moti- 
vate students  to  learn,  to  create  the  oppor- 
tunities to  learn,  and  to  foster  or  provide  a 
good  atmosphere  in  which  to  do  so.  In  or- 
der to  ensure  that  learning  does  indeed 
take  place.  We  have  to  be  quite  careful  to 
monitor  our  own  work,  and  be  careful  to 
take  the  guesswork  out  of  our  work.  By 
checking  understanding,  by  correcting  and 
eliciting , we  can  actually  see  what  is  being 
learned  as  it  happens,  or  at  the  end  of  the 
day  when  we  are  reflecting. 

I would  like  to  express  my  thanks  to  the  stu- 
dents and  teachers  at  Teacher  Training  Inter- 
national in  London  where  I attended  a TEFL 
course  that  taught  me  so  much , in  particular  to 
understand  and  use  some  of  the  techniques  and 
strategies  I have  discussed  in  this  article . 


ERIC 


^BDlome  11:  Winter  1998 


vi 


828 


ADVENTURES  IN  ASSESSMENT 


Volume 
11  • 32 


Assessment  tools  in  the  multi-level  ESOL  classroom 

The  More  Things  Change, 

the  More  They  Seem  to  Stay  the  Same 


by 

Maria  Elena 
Gonzalez 


Adult  Literacy 
Resource  Institute, 
SABES 
Boston,  MA 


Last  Fall  I returned  to  the  classroom 
for  the  first  time  in  five  years.  My 
absence  from  the  classroom  corre- 
lates with  my  time  in  staff  and  program 
development,  during  which  I undertook  a 
different  kind  of  teaching,  that  of  presenta- 
tions or  trainings  for  my  peers. 

During  that  time  I thought  a lot  about 
what  constitutes  good  teaching  and  learn- 
ing while  observing  and  talking  to  teach- 
ers. The  conversation  had  been  getting  a 
little  too  abstract  for  me,  however,  a bit  too 
removed  from  what  I had  actually  experi- 
enced in  the  classroom.  So  I chose  as  my 
own  staff  development  project  for  the  year 
to  teach  ESOL  once  a week  at  a small  com- 
munity-based organization  near  my  home. 

THE  ENGLISH  CLASS 

"La  clase  de  Ingles,"  as  my  students  re- 
ferred to  the  class,  was  entirely  composed 
of  women,  mostly  from  the  Dominican  Re- 
public. They  averaged  about  eight  per 
night,  but  always  began  at  about  20  during 
the  first  weeks  of  each  cycle.  A wonderful 
thing  about  this  class  was  that  it  offered 
childcare  to  each  woman  (or  man)  who 
came.  The  class  was  never  designed  to  be 
just  for  women,  but  available,  no-cost  child 
care  was  definitely  a big  draw  for  the  im- 
migrant women  who  lived  nearby. 

A feature  of  the  class  that  had  attracted 
me  initially  also  proved  a challenge  as  time 
went  on.  The  class  was  offered  twice  a 
week  but  the  teaching  was  shared  between 
me  and  another  teacher.  I taught  one  night 
and  she  taught  the  other,  which  worked 


wonderfully  for  our  schedules,  but  made 
it  challenging  to  plan  classes  together.  We 
did  meet  regularly  with  the  counselor,  to 
talk  about  the  students  and  to  plan  initial 
assessment  at  the  beginning  of  each  cycle. 

INITIAL  ASSESSMENT 

The  first  planning  session  was  at  the  ALRI 
library  where  I offered  a sample  of  various 
assessment  tools  and  past  copies  of  Adven- 
tures in  Assessment.  I also  had  samples  of 
tools  I had  used  as  an  ESOL  teacher,  in- 
cluding one  I had  designed  for  an  ad- 
vanced class  that  never  took  place.  This 
self-assessment  was  modeled  on  one  de- 
veloped by  teachers  at  Read/Write/Now 
Adult  Learning  Center  in  Springfield,  Mas- 
sachusetts. I always  liked  how  their  assess- 
ment is  based  on  principles  of  whole 
language  and  referred  back  to  it  several 
times  over  the  years.  It  is  not  so  much  one 
tool,  but  a series  of  tools  that  are  done  over 
a whole  cycle  of  classes.  (You  can  find  it 
and  an  accompanying  article  about  it  in 
Adventure  in  Assessment , Vol.  1.) 

The  version  I developed  focuses  on  a 
learner  self  assessment  of  the  different 
ways  s/he  may  be  using  English  across  the 
four  major  skill  areas  of  comprehension, 
speaking,  reading  and  writing  (see  "ESOL 
Assessment"  tool  at  the  end  of  this  article). 

The  learner  not  only  lists  those  areas  in 
which  s/he  has  difficulty,  but  also  thinks 
about  the  ways  in  which  s/he  already 
knows  and  uses  English. 

In  the  first  session,  for  example,  the 
learner  fills  in  the  following  sentence:  I 
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understand  English  at  (work,  my  children's 
school,  health  clinic,  etc.)  by  checking  off  one 
of  four  categories:  a little,  some,  a lot,  need 
more. " The  same  is  true  for  the  skill  areas 
of  speaking,  reading  and  writing.  The  stu- 
dent is  then  invited  to  look  over  what  s/he 
has  checked  off  and  notice  the  different 
ways  s/he  may  already  understand,  speak, 
read,  and  write  English. 

After  the  student  has  a sense  of  his/her 
areas  of  strength,  then  s/he  chooses  one 
area  from  each  skill  category  to  work  on  in 
class.  For  example,  a student  may  choose 
to  focus  on  "understanding  English  at 
church,"  "speaking  at  the  children's 
school,"  "reading  in  English  for  a job  ap- 
plication," "writing  notes  to  teachers."  The 
end  goal  is  that  the  learner  will  come  up 
with  an  individual  plan  for  the  class. 

MIXED  RESULTS 

As  I write  this  article  and  look  over  the 
assessment  forms  filled  out  by  the  learners, 
I realize  that  they  did  give  us  some  of  the 
information  we  wanted,  such  as  literacy 
levels  and  a broad  survey  of  the  learners' 
use  and  understanding  of  the  English  lan- 
guage. Yet  I was  disappointed  because  it 
failed  to  give  us  and  the  learners,  clear  and 
definitive  information. 

First,  the  design  of  the  tool  is  a bit  un- 
wieldy: it's  too  long  and  has  too  many 
crooked  lines  for  the  learners  to  navigate 
easily.  Second,  even  though  I explained 
each  category  in  Spanish  and  invited  the 
women  to  answer  in  Spanish,  it  was  still 
difficult  for  the  learners  to  understand 
how  the  form  was  organized.  It  is  some- 
what artificial  to  separate  listening  and 
speaking  skills,  especially  for  beginning 
students.  It  was  even  more  difficult  to  dif- 
ferentiate between  "a  little,"  "some,"  "a 
lot,"  and  "need  more."  As  an  ESOL 
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teacher,  I usually  don't  get  into  adverbs  of 
frequency  until  well  into  a cycle  and  nor- 
mally not  with  absolute  beginners.  The  dif- 
ficulty in  this  case  was  compounded  by  the 
fact  that  "poquito"  and  "algo"  ("a  little" 
and  "some,"  respectively)  mean  the  same 
thing  in  Spanish. 

Clearly,  this  tool  would  work  better 
with  more  advanced  learners,  for  whom 
I had  designed  it  in  the  first  place.  Even 
with  an  advanced  group,  though,  there 
needs  to  be  plenty  of  time  allotted  for 
learners  to  fill  it  out,  ideally  with  the  help 
of  more  than  just  one  teacher.  The  learners 
need  someone  to  walk  them  through  the 
instrument  and  to  talk  to  them  about  what 
they  check  off  and  why. 

My  co-teacher  and  I still  used  the  results 
of  this  initial  assessment  tool  to  learn  about 
each  student's  literacy  levels  as  well  as  to 
get  a general  idea  about  which  topics 
learners  were  interested  in.  For  the  end- 
of-cycle  evaluation,  however,  I wanted  a 
tool  that  would  be  more  interactive  and 
dynamic,  something  that  would  jog 
people's  memory  about  what  had  actually 
taken  place  in  my  Monday  class. 

END  OF  FIRST  CYCLE  EVALUATION 

On  newsprint  paper,  I wrote  the  theme  for 
each  class  next  to  the  date  it  occurred. 
Samples  of  the  lesson  were  stapled  on  the 
left  hand  side.  Each  student  was  given  five 
stars  to  place  next  to  their  favorite  lessons. 

I left  space  on  the  right  hand  side  for  writ- 
ten comments  which  they  could  write  in 
Spanish  if  they  wanted.  Once  I explained 
the  process  and  gave  them  their  stars,  I left 
the  room  for  as  long  as  they  needed. 

The  women  liked  the  task  and  gave  me 
some  predictable  as  well  as  unpredictable 
comments.  For  example,  they  rated  highly 
a couple  of  lessons  that  I thought  had 
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the  space  and 
perhaps  even  the 
confidence  of 
telling  me  what 
they  needed  to 
learn  better. 


bombed.  They  also  wrote  some  positive 
comments  about  me  as  the  teacher,  which 
is  typical  of  many  ESOL  classes,  but  those 
comments  were  also  peppered  with  some 
positive  criticism,  such  as  "I  would  like  for 
the  next  class  to  start  like  if  we  were  kids, 
starting  with  simpler  sentences,  at  a lower 
level." 

When  I asked  for  clarification  of  their 
comments,  a couple  of  learners  told  me  I 
had  gone  a bit  too  fast  for  them,  that  I 
erased  things  too  quickly  from  the  board, 
and  that  they  wanted  more  traditional 
types  of  lessons.  Clearly,  doing  the  evalua- 
tion had  given  the  learners  the  space  and 
perhaps  even  the  confidence  of  telling  me 
what  they  needed  to  learn  better.  The  ulti- 
mate, albeit  small  sign  of  their  progress 
was  the  one  comment  actually  written  in 
English:  "I  liked  is  (sic)  class  because,  se 
compartia  (we  shared)..." 


SECOND  CYCLE 

For  the  initial  assessment  of  the  second 
cycle,  we  decided  to  totally  change  gears 
from  what  we  had  done  in  the  first  cycle. 
This  time  we  decided  to  separate  the  as- 
sessment into  three  distinct  parts:  the  in- 
take (with  the  personal  information),  the 
literacy,  and  the  goal  setting.  For  the  first 
two  parts  we  used  an  assessment  tool  de- 
veloped by  the  state  of  New  York  for  use 
in  their  adult  basic  education  ESOL  classes. 
It  comes  in  various  languages  and  tests  lit- 
eracy through  a series  of  short  readings  in 
English  at  various  levels.  After  reading 
them,  learners  are  encouraged  to  write  a 
short  paragraph  about  themselves.  Unfor- 
tunately, doing  the  personal  piece  and  the 
goal  setting  (more  about  that  later)  took  all 
of  the  first  day,  and  the  following  class  was 
cancelled  due  to  a holiday.  We  lost  mo- 
mentum and  never  went  back  to  the  read- 
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ing  piece. 

We  did  get  a small  flavor  of  what  it 
would  be  like  to  do  the  entire  assessment 
from  one  learner  who  had  done  the  Span- 
ish version.  She  actually  wrote  an  entire 
paragraph  where  she  revealed  she  only 
had  a second  grade  education  (she  had  left 
that  question  blank  in  the  first  part  of  the 
assessment).  She  wrote: 

"si  llo  quiero  aprender  por  que  me  es  mui 
dificil  cuando  sorgo  a buscar  trabajo.  cuando 
voi  a los  apoime  que  tengo  que  usar  urn  tercera 
persona  isi  no  aparese  interprete.  me  sucedio 
unabes,  tube  tres  hora  es  perando  por  un 
interprete.  me  asido  dificil  por  mi  nibel  de 
estudio  segundo  de  primaria  gracia" 

("yes,  I want  to  learn  because  it  has  been 
very  hard  when  I go  looking  for  work.  When  1 
go  to  "apoime"  (appointment)  I have  to- use. a 
third  person  if  there  is  no  translator.  It  hap- 
pened that  one  time  I was  three  hours  waiting 
for  an  interpreter.  It  has  been  hard  because  my 
level  of  studies  is  second  grade  primary.  Thank 
you.) 

In  this  one  example,  we  can  see  how  the 
tool  is  meant  to  work  from  soliciting  per- 
sonal data  to  actually  gauging  reading  and 
writing  levels. 

BREAKING  THE  ICE,  SETTING  GOALS 

For  the  goal  setting,  I decided  to  do  a 
variation  on  a "breaking  the  ice"  group  ac- 
tivity Elsa  Auerbach  used  a long  time  ago 
and  which  I have  used  many  times  since. 
The  learners  split  into  two  lines  facing  each 
other.  I brought  in  my  tape  recorder  with 
contemporary  African  music  and  told  the 
students  they  had  to  walk  or  dance  mov- 
ing to  their  left  until  I stopped  the  music. 
Then,  and  only  then,  could  they  stop  and 
face  the  person  in  front  of  them.  They  had 
to  interview  the  other  student  and  ask 
them  their  name,  their  hometown  (since 
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most  were  from  the  Dominican  Republic), 
how  many  years  they  were  in  this  country, 
and  why  they  wanted  to  learn  English.  I 
told  them  to  pay  special  attention  to  the 
latter  question  since  they  would  be  asked 
to  introduce  me  to  one  other  person  at  the 
end  of  the  activity.  It  was  a lot  of  fun  to  see 
the  women  dancing,  including  the  grand- 
mothers, to  the  beat  of  the  infectious  Afri- 
can beat.  I let  it  go  on  for  a couple  of 
rounds  and  when  we  stopped,  I asked 
each  learner  to  introduce  me  to  one  of  the 
people  they  had  talked  to  and  tell  me  why 
they  wanted  to  ieam  English. 

As  each  woman  introduced  another,  I 
wrote  what  each  said  on  newsprint.  We 
went  over  it  the  next  class,  adding  the 
voices  of  those  who  had  not  been  present. 

I then  asked  the  learners  to  look  for  com- 
mon themes  or  goals  in  the  list  of  reasons 
why  they  wanted  to  learn  English.  It  took 
us  a while  to  make  the  distinction  between 
general  goals  like  "I  want  to  learn  English 
because  I like  it,"  to  more  concrete  goals 
like  "I  need  English  when  I go  to  the  doc- 
tor." We  came  up  with  a short  list  that  in- 
cluded doctors,  children's  schools,  work, 
shopping,  and  citizenship  tests.  The  group 
finally  decided  to  start  with  doctors  as  a 
class  topic  for  that  cycle. 

CYCLE  EVALUATION 

The  last  day  of  the  cycle  was  suddenly 
upon  us  and  the  agency  organized  a cel- 
ebration to  which  members  of  the  Board 
of  Directors  were  invited.  I had  to  design 
something  that  could  be  completed  in  fif- 
teen minutes  or  less,  so  I asked  myself 
what  did  I really  want  to  know?  What 
questions  would  give  me  the  bare  bones  of 
information  I needed  to  evaluate  the  class 
and  plan  for  the  next  cycle?  The  questions 
also  had  to  be  as  simple  and  direct  as  pos- 
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sible  to  minimize  the  amount  of  explaining 
I would  have  to  do.  The  following  ques- 
tions, typed  out  in  big  print  on  a single 
sheet,  fulfilled  those  requirements  with 
fairly  good  results: 

1)  This  cycle  I learned: 

2)  I used  my  English  at: 

3)  Next  cycle  I want  to  learn  about: 

4)  The  teachers  help  me  to  learn  when 
they: 

5)  The  teachers  do  not  help  me  to  learn 
when  they* 

For  the  first  question,  I asked  them  to  be 
as  specific  as  they  could,  encouraging  them 
to  write  down  anything  at  all  that  they  re- 
membered from  the  class.  Many  wrote  the 
parts  of  the  body  we  had  been  learning 
and  showed  themselves  in  a very  tangible 
way  that  indeed  they  had  "learned  some- 
thing." I was  hoping  for  the  same  degree  of 
specificity  in  the  second  question,  "I  used 
my  English  at..."  but  it  was  too  hard  for 
most  to  go  beyond  a list  of  "work,  bus,  etc. 
"I  tried  to  get  verbal  anecdotes  from  some 
as  I went  around.  One  woman  wrote  "may 
(sic)  house"  as  a place  where  she  had  used 
her  English.  Since  I knew  she  had  children, 
I kidded  her  by  saying,  "Oh,  Alba,  you  are 
now  talking  to  your  kids  in  English."  "No, 
with  my  neighbor,"  she  replied,  then  told 
me  how  she  translated  a letter  from  the 
clinic  to  her  neighbor  and  felt  that  she  had 
been  able  to  do  so  because  of  the  health 
theme  in  our  classes.  This  is  the  kind  of  de- 
tail I wanted  but  which  is  hard  to  get  from 
a written  evaluation  unless  you  ask  the 
"right"  question  and  have  the  time  to  prod. 

Question  #3  was  also  helpful  in  giving 
us  information  about  the  next  cyle  but  #4, 
"The  teachers  help  me  to  learn  when 
they..."  was  a mixed  bag  and  #5  did  not 
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work  at  all.  I speculate  that  either  it  was 
too  close  to  #4  (the  same  words,  except  for 
"do  not")  or  it  would  have  meant  being 
critical  of  the  teachers,  which  adult  learn- 
ers usually  don't  like  to  do.  Even  after  ex- 
plaining the  question  in  Spanish,  most 
wrote  variations  on  "my  teachers  are 
good"  and  a couple  showed  that  they  had 
not  undestood  the  question  at  all.  Overall, 

I liked  the  simplicity  of  this  form  but 
would  have  liked  more  time  to  really  get 
into  the  questions  with  the  students  and 
try  to  get  them  to  give  me  (and  them- 
selves) as  much  information  as  possible. 

I developed  an  even  simpler  final  evalu- 
ation tool  for  the  end  of  the  third  cycle.  We 
simply  asked  the  students  what  they  liked 
most  about  the  class  and  what  they  had 
not  liked  and  any  recommendations  for  a 
future  class.  We  got  much  more  direct  in- 
formation such  as  recommendations  to  use 
the  book  in  class  (we  had  picked  a self-study 
text  of  English  for  Spanish  speakers)  to  the 
by  then  ubiquitous  "don't  erase  so  fast." 

THIRD  CYCLE 

Simplicity  and  speed  were  the  motivating 
factors  for  the  short  third  cycle.  Because 
we  already  had  information  about  return- 
ing students,  I had  come  up  with  a "test" 
that  included  materials  from  the  last  cycle, 
mainly  in  response  to  the  women's  request 
for  more  traditional  exercises  such  as  dic- 
tation and  quizzes.  It  worked  very  well  be- 
cause it  provided  a sense  of  continuity  for 
those  who  had  been  coming  most  of  the 
year  and  it  was  in  an  assessment  format 
that  felt  familiar  to  them.  For  me,  the 
teacher,  it  was  also  a familiar  way  in  which 
to  assess  how  folks  had  progressed  and 
whether  they  had  engaged  with  the  mate- 
rial. 


THE  "JOHN'S  TEST" 

I chose  the  true-and-tried  "John's  test"  for 
the  initial  evaluation  of  new  learners  be- 
cause we  needed  a tool  that  would  give  us 
the  maximum  amount  of  information  in 
the  quickest  amount  of  time.  There  were 
only  two  of  us  to  assess  20  people,  half  of 
whom  were  brand  new.  My  co-teacher 
took  over  the  assessment  of  returning 
learners,  while  I met  with  the  new  learners, 
which  included  two  young  men.  We  only 
had  about  two  hours  in  which  to  do  it  all 
and  the  John's  test  was  perfect  for  that 
kind  of  situation.  I could  conduct  the  oral 
proficiency  piece,  for  example,  while  one 
student  was  filling  out  the  intake  piece  and 
another  could  be  finishing  Part  II,  the  lit- 
eracy screening. 

The  John's  test,  as  many  teachers  of 
ESOL  to  adults  know,  was  widely  used  in 
programs  until  fairly  recently.  It  was  one 
of  the  first  tests  (1975)  originally  designed 
to  test  English  oral  proficiency  in  adults  at 
basic  education  programs.  It  was  also  not 
copyrighted  which  further  encouraged  its 
use  but  mostly,  it  is  a quick,  easy-to-use 
tool  that  concentrates  on  testing  listening 
and  speaking  skills.  Through  the  years, 
teachers  have  adapted  it,  mostly  to  include 
a reading/writing  component. 

The  version  I used  is  so  changed  that  it 
hardly  can  be  called  the  John's  test.  For  one 
thing,  "John"  is  gone.  In  the  original  ver- 
sion, a series  of  seven  pictures  showed  a 
day  in  the  life  of  John.  Learners  were  ex- 
pected to  try  to  describe  what  was  happen- 
ing with  John  in  those  pictures.  At  the 
Boston  Workplace  Collaborative,  where  I 
first  taught  ESOL,  we  got  rid  of  John  and 
used  the  personal  data  questions  as  a 
"test"  of  oral  profiency.  A "0"  indicated  no 
comprehension,  while  a "3"  indicated  an 
advanced  level.  A two-part  literacy  screen- 
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ing  followed  that  goes  from  very  simple 
matching  of  words  to  pictures  of  common, 
everyday  things  like  "fire,"  "house,"  etc., 
to  more  advanced  tasks  of  reading  a short 
letter  and  answering  comprehension  ques- 
tions. 

It  had  been  many  years  since  I had  last 
used  it,  but  I found  it  worked  fairly  well  in 
answering  the  kinds  of  questions  we  had 
about  a brand  new  learner.  Could  they  un- 
derstand and  answer  simple,  personal 
questions  such  as  "what  is  your  name, 
how  many  years  of  formal  schooling  have 
you  had  in  your  country,  can  you  read  or 
write  at  all,  can  you  read  in  English  at  all? 

It  also  helped  us  deal  with  our  biggest 
challenge,  which  was  to  assess  folks  as 
quickly  as  possible.  Like  most  assessment 
tools,  it  was  less  effective  in  giving  us  ac- 
curate levels  of  people's  knowledge  and 
skills  but  it  did  give  us  elementary  knowl- 
edge about  each  student. 

THE  LESSON  IN  ALL  THIS? 

This  is  the  question  I always  ask  myself  af- 
ter I've  been  faced  with  a challenge  where 
I have  had  to  reevaluate  assumptions  and 
change  a course  of  action.  I think  the  big- 
gest realization  for  me  after  this  experi- 
ence, and  which  I had  forgotten  in  my 
years  out  of  the  classroom,  was  how  time- 
consuming  assessment  is.  It  was  the  big- 
gest factor  in  my  decision  to  go  from  using 
a "Cadillac"  of  a tool  to  a true-and-tried 
placement  test.  Ideally,  initial  assessment 
should  be  done  at  a special  time  with  up  to 
thirty  minutes  allotted  per  learner,  so  there 
is  enough  time  at  the  end  for  the  teacher  or 
counselor  to  talk  to  the  student  about  the 
process. 

A casualty  of  the  time  crunch  is  on- 
going assessment.  There  simply  wasn't 
enough  time  in  our  three  cycles  to  include 


a formal  assessment  even  though  there 
were  a couple  of  quizzes  and  frequent 
checking  in  between  my  co-teacher,  the 
counselor,  and  me.  For  the  most  part  those 
conversations  tended  to  focus  on  other  ba- 
sic needs  our  students  had,  many  of  whom 
were  facing  cuts  in  social  welfare  benefits 
such  as  food  stamps.  This  was  especially 
disturbing  for  us  when  we  realized  that  for 
at  least  one  mother,  the  snack  we  gave  to 
her  kids  during  childcare  was  her  biggest 
incentive  for  attending  class. 

A colleague  asked  me  why  I tried  differ- 
ent tools  instead  of  sticking  with  one 
throughout  the  year  to  see  how  it  worked 
with  different  learners.  It  struck  me  that  I 
had  not  even  considered  that  option, 
which  is  a good  one  given  that  in  my  expe- 
rience, every  learner  and  situation  in  ESOL 
is  different.  I believe  my  motivation  for 
changing  tools  was  simply  the  need  to  find 
one  that  fit  certain  criteria  that  were  not 
very  clear  at  the  beginning  but  which 
evolved  as  the  year  went  by. 

Through  this  experience  I have  come  to 
believe  that  developing  criteria  for  assess- 
ment should  not  only  include  standards 
for  measuring  skills  and  levels  but  other 
factors  as  well,  such  as  composition  of  the 
class,  frequency  of  meetings  and  length  of 
cycles,  and  ultimately,  what  really  makes 
sense  to  do.  I had  started  out  by  being 
overly  concerned  with  assessing  people's 
skill  levels  in  English  and  getting  a picture 
of  literacy  in  their  lives.  Those  were  good 
goals  but  I needed  to  start  with  some 
simple,  basic  questions  such  as,  what  do 
we  really  need  to  know  about  a new  stu- 
dent when  they  walk  through  the  door, 
how  can  we  assess  returning  students  (es- 
pecially when  there  are  no  other  levels), 
what  is  realistic  given  the  time  available 
for  the  assessment  as  well  as  the  class  it- 
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self,  what  can  we  find  out  about  a previous 
class  to  inform  the  next  one,  what  do  learn- 
ers want  to  learn. 

Ultimately,  I don't  believe  any  one  lan- 
guage assessment  tool  can  give  us  the 
whole  picture  of  a learner.  At  best,  an  ini- 
tial assessment  tool  can  give  the  teacher  a 


snapshot  of  the  student  and  the  final 
evaluation  is  the  end  result  of  an  ongoing 
conversation  about  what  is  going  on  in 
class,  what  needs  to  be  better,  and  what  the 
learner's  needs  are.  Getting  answers  to  some 
of  these  questions  became  more  important 
to  me  than  the  exact  tool  I was  using. 
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SELF  ASSESSMENT  TOOL 


I understand  English  at: 

work 

my  children's  school 
the  health  clinic 
church 

the  supermarket 
the  clothing  store 
the  shoe  store 
my  school 
parties 

immigration  office 

unemployment  office 

buses/train 

television 

radio 

community  meetings 

I speak  English  at: 


work 

my  children's  school 

the  health  clinic 

church 

stores 

my  school 

the  police 

utility  company  (gas, 
electricity,  phone,  cable  TV 
immigration  office 
unemployment  office 
buses/ train 
community  meetings 
Other 


Modeled  after  ReadWriteNow  (ALA,  Volume  1 ) 
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Look  at  the  different  ways  you  may  be  reading  and  writing  in  English 


I can  read  in  English: 


a little 


some 


a lot 


need  more 


name  and  address 
telephone  numbers 
instructions  for  medicine 
magazines 
newspapers 
TV  guide 

notes  from  children's  school 

job  announcements 

bills 

instructions  at  work 
job/credit  application 
bank  slips 
children's  books 
Bible/religious  books 
ads  for  apartments 
letters 

signs  in  the  stores 

papers  from  immigration 

notice  about  community  events 

local  newspaper 

comic  books 

other 


I can  write  in  English: 


a little 


some 


a lot 


need  more 


name  and  address 
job  application 
credit  application 
letters 

notes  to  my  kids'  teachers 

poems 

stories 

other:  (please  list) 
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Do  you  like  to  read  and  write  in  your  native  language? 

If  you  do,  what  do  you  read  and  write? 

□ letters 

□ newspapers 

□ magazines 

□ Bible/religious  books 

□ romances 

□ novels 

□ comic  books 

□ detective  stories 

□ photonovelas 

□ other 

Do  you  speak  to  your  children  in  your  native  language? 

Look  over  your  list  for  a few  minutes  and  notice  the  different  ways  that 
you  already  may  understand,  speak,  read  and  write  English,  even  if  it  is 
just  a little  bit.  Are  there  any  surprises? 

Now  look  over  the  right  column  where  you  need  more  help.  Choose  one 
(1)  area  from  each  category.  Example: 

I want  to  understand  English  at  the  community  meetings. 

I want  to  speak  English  at  my  children's  school. 

I want  to  read  in  English  a job  application. 

I want  to  write  in  English  notes  to  my  kids'  teachers. 

Now  try  it  for  yourself. 

I want  to  understand  English i 

I want  to  speak  English i 

I want  to  read  in  English . 

I want  to  write  in  English t 

These  are  my  personal  goals  for  this  class. 
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ENGLISH  CLASS  CYCLE  EVALUATION 


This  cycle  I learned: 


used  English  at: 


Next  cycle  I want  to  learn  about: 


The  teachers  help  me  to  learn  when  they: 


The  teachers  do  not  help  me  to  learn  when  they: 
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Is  On-Going  Assessment 
Fully  Learner-Centered? 


by 

Linda  A. 
Gosselin 


Quinsigamond 
Community  College 
Worcester,  MA 
Clinton  Adult 
Learning  Center 
Clinton,  MA 
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During  1997/1998, 1 was  a member 
of  the  group  that  developed  the 
Framework  for  Adult  ESOL  in 
Massachusetts.  One  of  the  cross-cutting 
themes  in  that  document  is  assessment. 

I had  volunteered  to  examine  and  write 
something  up  on  assessment  for  the  draft. 

Working  alone  and  with  few  guidelines 
to  follow,  I decided  that  a useful  resource 
to  aid  me  in  my  task  would  be  the  Adven- 
tures in  Assessment  series.  Since  the  articles 
had  been  written  by  practitioners  from 
around  the  state,  I thought  I would  get  a 
good  sampling  of  what  is  being  done  in 
ESOL  assessment  in  a variety  of  programs. 

I could  use  that  information,  along  with 
my  own  experiences  with  assessment  in  20 
years  of  teaching  ESOL,  to  write  a piece  on 
assessment. 

Since  there  are  standardized  and  program- 
designed  tests  for  initial  and  exit  testing, 

I chose  on-going  assessment  as  the  main 
focus  of  my  piece.  At  that  time  I chose  to 
outline  the  various  forms  of  on-going  as- 
sessment, give  some  examples,  and  stress 
the  efforts  of  practitioners  to  make  on-go- 
ing assessment  student-centered. 

In  this  journal,  I further  explore  on-go- 
ing assessment  and  what  I learned  from 
reading  all  the  ESOL  articles  in  prior  vol- 
umes of  Adventures  in  Assessment. 

MY  PREVIOUS  UNDERSTANDING 
OF  ON-GOING  ASSESSMENT 

Before  reading  all  of  the  articles  in  Adven- 
tures in  Assessment,  I had  defined  on-go- 
ing assessment  as  examining  and 
documenting  learner  progress  at  certain  in- 


tervals in  the  learning  process.  In  the  pro- 
gram I teach  in  at  Quinsigamond  Commu- 
nity College,  we  had  designed  two  assess- 
ment forms  for  on-going  assessment  (see 
'Bringing  Learners  into  Goal-Setting/ 
Burke,  Gosselin,  and  Shea,  Vol.  8).  One 
form  is  for  students  to  reflect  and  comment 
on  their  own  progress.  The  other  form  is 
for  the  teacher  to  comment  on  each 
learner's  progress.  We  have  also  used  port- 
folios — usually  folders  of  samples  of  a 
learner's  work  and  copies  of  the  above 
mentioned  assessment  forms  — which 
were  to  document  the  learner's  progress. 
This  documentation  has  been  used  to  assist 
us  in  judging  when  a learner  is  ready  to 
move  on  to  the  next  level  class  and  to  pro- 
vide the  teacher  of  the  next  level  with  a 
record  of  the  learner's  work. 

A NEW  UNDERSTANDING  OF 
ON-GOING  ASSESSMENT 

Many  articles  in  the  assessment  journals 
defined  on-going  assessment  in  a way 
similar  to  my  definition  above.  But  there 
were  also  articles  that  defined  on-going  as- 
sessment as  evaluation  of  the  class  and  ac- 
tivities taking  place  in  class.  The  assess- 
ment tools  mentioned  in  these  articles  ask 
students  to  assess  class  activities  as  to 
whether  the  students  enjoyed  them  and 
whether  the  activities  were  of  value  to  the 
students  in  helping  them  to  make  progress. 

Like  any  good  instructor,  I want  to  be 
responsive  to  the  needs  of  my  learners.  So 
I often  ask  for  feedback  from  my  students 
about  activities  we  are  doing,  especially  if 
the  activities  are  new  ones  for  the  class. 
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Also,  I have  always  tried  to  encourage  my 
students  to  speak  up  if  they  feel  something 
is  not  helpful  or  if  they  want  the  class  to 
change  in  some  way.  But  I never  thought 
of  having  a more  formal  assessment  tool  of 
class  activities  for  students  to  complete  as 
described  in  some  of  the  articles  I had  read. 

Some  practitioners  talked  about  using 
'oral  feedback'  to  ask  students  how  they 
feel  about  the  class.  Caroline  Gear  (Vol . 8, 
p.  46)  states: 

" Asking  learners  about  their  progress  and 
satisfaction  with  the  program  enables  learners 
to  begin  thinking  about  how  they  team  and 
what  works  best  for  them." 

Some  instructors,  including  Gear,  record 
a learner-generated  list  of  activities  on  the 
board  or  on  a form  and  the  students  are 
asked  to  rank  them  numerically  or  with  a 
happy,  neutral,  or  unhappy  face,  according 
to  their  enjoyment  of  each  activity.  Other 
articles  talked  about  having  learners  keep 
a weekly  log  and  write  how  they  felt  about 
class  activities  done  in  the  week. 

By  doing  this  type  of  assessment,  learn- 
ers can  identify  which  activities  they  like 
and  which  ones  they  find  helpful  to  their 
learning.  It  provides  an  instructor  with 
some  documentation  of  learner  preferences. 

Both  types  of  on-going  assessment 
above  are  vital  to  an  effective  ESOL  class- 
room. They  inform  instructors  which  ac- 
tivities their  learners  find  most  helpful  and 
allow  instructors  to  plan  lessons  accord- 
ingly. It  also  provides  a record  of  progress. 

REFLECTIONS  ON  LEARNER-CENTERED 
ASSESSMENT 

A common  theme  throughout  this  journal 
series  has  been  learner-centered  assess- 
ment. Looking  at  the  two  types  of  on-going 
assessment  I mentioned  above,  I wondered 
how  much  they  include  learners  in  the 
process  and  at  what  point  in  the  process. 


I reviewed  everything  that  had  been 
written  in  past  issues  on  student  involve- 
ment in  on-going  assessment.  In  Volume  1, 
learner-centered  assessment  was  defined 
as  "a  collaborative  relationship  among 
learners  and  program  staff  in  determining 
the  goals,  texts,  and  contexts  of  assessment, 
as  well  as  judging  its  outcomes"  (p.  ii). 

If  I interpret  this  quote  correctly,  it 
means  that  learners  should  be  involved  in 
the  entire  planning  process  of  the  purpose 
and  form  of  on-going  assessment.  Learners 
should  be  full  partners  in  deciding  the 
goals  of  on-going  assessment.  Tney  should 
be  involved  in  deciding  how  and  when 
progress  will  be  measured.  They  should 
take  part  in  deciding  which  class  activities 
will  be  most  helpful  to  their  progress. 

In  examining  my  own  on-going  assess- 
ment practices,  I can  see  changes  that  I can 
make  so  that  my  students  have  greater 
ownership  of  the  process.  For  example,  at 
the  beginning  of  a school  year,  I can  gener- 
ate a list  of  common  class  activities  in  col- 
laboration with  my  students.  My  students 
can  tell  me  which  ones  they  prefer.  Such  a 
list  might  include  such  things  as  dictation, 
being  read  aloud  to,  doing  small  group 
work,  watching  a video,  and  taking  field 
trips.  Using  the  input  I receive  from  my 
students,  I can  plan  my  lessons  to  include 
the  preferred  activities. 

At  regular  intervals,  the  students  could 
assess  the  activities  as  to  whether  they 
have  been  helpful  to  the  students'  progress. 
Changes  in  activities  could  be  made  where 
desired  or  deemed  necessary. 

I could  also  conduct  a learning-style 
inventory  with  my  students  to  help  them 
understand  which  types  of  activities 
would  be  better  for  them,  based  on  their 
predominant  learning  style.  Such  an  activ- 
ity would  help  them  understand  why  they 
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Students  could 
choose  the  on-going 
assessment  tools 
we  will  use  to 
document  progress. 
At  the  end  of  a 
cycle,  they  could 
evaluate  whether 
the  tools  we  used 
really  reflected 
the  progress  they 
thought  they  made. 


seem  to  prefer  certain  class  activities  over 
others. 

Also,  the  students  and  I could  discuss 
their  goals  for  a certain  period  of  class  time 
and  collaboratively  decide  how  and  how 
often  we  will  document  on-going  learner 
progress  (keeping  in  mind  any  program 
requirements). 

Rather  than  have  the  students  simply  fill 
out  a self-assessment  tool  and  call  it 
learner-centered,  students  could  choose  the 
on-going  assessment  tools  we  will  use  to 
document  progress.  At  the  end  of  a cycle, 
they  could  evaluate  whether  the  tools  we 
used  really  reflected  the  progress  they 
thought  they  made.  If  the  students  do  not 
feel  comfortable  thinking  up  ways  to  docu- 
ment progress,  I could  show  them  com- 
monly used  samples  of  on-going  assess- 
ment tools  and  they  could  choose  the  ones 
they  think  will  be  most  effective. 


CONCLUSION 

The  sub-title  of  this  journal  is  "Learner- 
Centered  Approaches  to  Assessment  and 
Evaluation  in  Adult  Literacy."  That  is  what 
we  as  teachers  have  been  striving  for  in  us- 
ing and  contributing  to  this  journal.  We 
must  continue  to  examine  our  own  on-going 
assessment  practices  and  ask  ourselves  some 
pertinent  questions.  How  much  do  we  really 
involve  our  learners  in  the  assessment  pro- 
cess? At  what  point  in  the  process  do  we  in- 
clude our  learners?  Is  the  degree  of  learner 
involvement  in  on-going  assessment  in  our 
programs  enough?  Are  there  ways  we  can 
provide  our  learners  with  greater  ownership 
in  the  process  of  choosing  and  designing  on- 
going assessment  tools? 

In  Volume  2,  p.  iv,  Loren  McGrail  stated, 

"If  all  we  do  is  substitute  new  multiple  mea- 
sures for  old  standardized  measurements  and 
monitor  student  progress  for  diagnostic  pur- 
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poses  in  terms  of  identifying  strengths  and 
weaknesses...,  we  will  not  have  created  a new 
paradigm.  If,  on  the  other  hand,  we  include 
learners  as  active  participants  at  the  center  of 
the  process  of  measurement,  as  'co-investiga- 
tors' in  determining  their  ovm  literacy  prac- 
tices, strengths  and  strategies  (Lytle,  1998), 
then  we  have  truly  engaged  in  alternative 
assessment." 

In  reading  through  all  the  volumes  of 
Adventures  in  Assessment,  I can  see  our  pro- 
cess of  growth  and  change  in  assessment 
and  clearly  see  that  we  have  come  a long 
way  in  improving  on-going  assessment  so 
that  it  is  more  meaningful  to  learners  and 
teachers  and  provides  for  greater  learner 
participation.  We  have  learned  a lot  from 
our  fellow  practitioners  who  have  experi- 
mented in  their  classrooms  and  who  have 
been  generous  enough  to  share  their  suc- 
cesses and  failures  so  that  we  could  learn 
from  them  and  grow  in  the  process  too. 

We  need  to  continue  to  examine  how  we 
can  include  our  students  as  full  partners  in 
on-going  assessment  at  all  stages  in  the 
process.  Then  we  can  fulfill  the  goal  that 
Loren  McGrail  set  for  assessment  in  adult 
education  in  Massachusetts:  learner-cen- 
tered on-going  assessment. 

In  the  introduction  to  Volume  10,  p.  4, 
Alison  Simmons  wrote,  "There  is  not  a 
clear,  bright  light  at  the  end  of  the  assess- 
ment tunnel  that  will  resolve  all  the  many 
dimensions  of  assessment  that  are  inherent 
in  our  classrooms  and  programs." 

What  she  says  is  very  true,  but  in  shar- 
ing with  each  other  in  this  unique  journal, 
we  can  help  light  the  way  as  we  navigate 
through  this  on-going  assessment  tunnel. 
In  doing  so,  we  can  guide  each  other  in  im- 
proving our  on-going  assessment  practices 
and  make  on-going  assessment  more 
learner-centered . 
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Assessment  and  Accountability 

A Modest  Proposal 


At  times  it  seems  that  everything 

there  is  to  say  about  testing  and  as- 
sessment in  adult  literacy  has  been 
said.  By  now,  practitioners  and  administra- 
tors alike  can  dte  the  shortcomings  of  stan- 
dardized tests  using  multiple  choice  formats 
and  are  familiar  with  the  inadequacy  of 
grade  levels  as  indicators  of  what  adult 
learners  know  and  are  able  to  do  . 

Yet,  pencil  and  paper,  multiple-choice 
tests  continue  to  be  used  not  only  as  place- 
ment instruments  but  as  measures  of 
learner  gains  and  evidence  of  program 
success.  Given  current  reporting  require- 
ments, their  use  is  likely  to  increase,  at 
least  in  the  near  future. 

From  the  perspective  of  programs,  there 
seem  few  viable  alternatives  that  would 
meet  the  information  needs  of  funders  in- 
terested in  reliable  data  that  indicate  how 
a program  is  doing  overall.  Portfolio  ap- 
proaches, for  example  — considered  the 
last  great  hope  a few  years  back  — have 
not  quite  matured  to  the  level  where  they 
might  be  used  as  a means  to  report  and  ag- 
gregate learner  gains  by  group  (although 
they  are  invaluable  as  evidence  of  indi- 
vidual learner  progress),  largely  because 
the  field  has  not  invested  in  the  develop- 
ment of  benchmarks  and  rubrics. 

Local  approaches  have  remained  just 
that,  local  approaches,  primarily  for  two 
reasons:  1)  there  has  not  been  enough  field 
testing  to  establish  the  reliability  of  these 
measures  and  2)  there  have  not  been  suffi- 
cient efforts  to  implement  alternative  assess- 
ments across  programs.  At  this  time,  it  is 
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easy  to  see  how  even  programs  that  have 
been  enthusiastic  about  developing  an  as- 
sessment system  that  captures  what  they 
consider  worthwhile  outcomes  are  becom- 
ing distressed  about  the  prospects  of  an 
alternative  system  being  able  to  rival  the 
standardized  tests  currently  in  fashion. 

ALL  IS  NOT  LOST 

Yet,  the  picture  is  not  as  dim  and  grim 
as  it  might  first  appear.  Indeed,  it  may  be 
premature  to  give  in  to  cynicism  ("it's  all  a 
sham  and  no  one  really  cares'7)/  paranoia 
("next  year,  all  funding  will  be  tied  to  the 
results  of  standardized  tests"),  and  paraly- 
sis ("in  the  end,  no  one  will  care  about  al- 
ternative assessment,  so  let's  just  sit  and 
wait  to  see  what  comes  down  the  pike"). 
Since  a Pollyanna  attitude  does  not  appear 
to  be  justified  either,  given  recent  legisla- 
tion, perhaps  it  is  time  to  take  an  existen- 
tialist perspective  where  we  commit 
ourselves  to  forge  ahead  although  (and 
even  because)  life  in  adult  literacy  does  not 
always  make  sense,  but  what  else  are  we 
going  to  do  to  stay  sane? 

Let's  ask  then  if  there  is  anything  posi- 
tive happening  in  assessment,  and  how 
we  can  help  shape  new  directions  on  the 
national  or  state  level,  while  continuing  to 
strive  for  sane  assessments  within  and 
across  local  programs. 

THE  FEDERAL  OUTCOME  REPORTING 
SYSTEM 

You  may  have  heard  that  the  U.S. 
Department  of  Education  has  mandated  a 
uniform  outcome-based  reporting  system 
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that  requires  that  all  states  send  data  for  all 
programs  funded  under  Adult  Basic  Edu- 
cation (ABE)  to  the  Department  of  Educa- 
tion in  Washington.  Assessments  for 
capturing  outcomes  must  be  "valid  and  re- 
liable." In  other  words,  they  must  either  be 
in  the  form  of  a standardized  test  (consid- 
ered reliable  by  definition)  or  by  some 
other  means  that  meet  these  requirements. 
States  (and  the  programs  they  fund)  will 
be  asked  to  report  "learner  gains"  in  read- 
ing, writing,  speaking,  and  listening  (and 
possibly  additional  skills  related  to  work- 
force development)  and  show  that  learners 
are  advancing  across  levels,  such  as  the 
Student  Performance  Levels  (SPL)  estab- 
lished for  ESL.  These  are  minimal  require- 
ments and  individual  states  can  define 
progress  in  various  ways  or  even  suggest 
additional  outcomes  as  evidence  of  literacy 
progress  and  program  success. 

To  understand  the  thinking  behind  the 
initiative,  it  is  important  to  keep  in  mind 
that  the  primary  focus  is  neither  curricu- 
lum reform,  nor  program  improvement 
(although  new  assessment  systems  are  of- 
ten used  for  these  purposes),  but  rather  an 
accountability  measure  to  bring  adult  lit- 
eracy in  line  with  the  requirements  of 
GPRA  — the  Government  Performance 
and  Results  Act.  GPRA  requires  that  all 
federal  agencies  have  to  show  that  they,  as 
well  as  the  agencies  and  programs  they 
fund,  are  achieving  results  or  else  risk  loss 
of  funding.  Since  the  focus  of  GPRA  is  on 
the  performance  of  the  overall  system 
(made  up  of  thousands  of  programs),  nei- 
ther the  federal  government  nor  the  states 
are  likely  to  pay  a great  deal  of  attention  to 
the  progress  made  by  any  given  learner  at 
any  given  site,  although  site  performance 
will  be  open  for  review  (think  standard- 
ized testing  in  K-12).  Rather,  funders  will 


want  to  know  how  a program  is  doing 
overall  (that  is,  whether  it  is  positively  af- 
fecting literacy  skills),  and  they  expect  to 
see  numbers  in  aggregate  (summarized) 
form. 

While  in  many  ways,  documenting  the 
kinds  of  outcomes  required  by  the  new  re- 
porting system  are  "doable"  (at  least  for 
programs  that  have  long  reported  literacy 
gains  for  a sample  of  their  students),  two 
dangers  loom  as  programs  try  to  show 
gains  for  all  students  (not  just  a sample) 
and  as  results  are  increasingly  tied  to  fund- 
ing. There  is  a risk  that  programs  will  be 
a)  tempted  to  manipulate  assessment  re- 
sults in  their  favor  and  b)  succumb  to  a 
practice  known  as  "creaming". 

MANIPULATING  ASSESSMENT  RESULTS 

Any  time  success  (and  subsequent  fund- 
ing) is  determined  by  the  data  a program 
reports,  there  are  concerns  about  adminis- 
trators "fudging  the  data."  For  example, 
programs  have  long  known  that  the  trick 
to  increasing  test  scores  is  to  NOT  prepare 
students  for  the  test,  but  rather  to  assess 
them  as  soon  as  they  walk  in  the  door.  This 
keeps  baseline  scores  artificially  low  and 
progress  is  inflated,  since  gains  are  due  to 
increases  in  test-wiseness,  rather  than  any 
real  gains  in  literacy  skills.  Although  this 
kind  of  manipulation  is  considered  unethi- 
cal, since  the  resulting  data  "lack  integ- 
rity". The  practice  is  nevertheless  quite 
commonplace  among  programs  pressured 
to  demonstrate  learner  progress  in  short 
amounts  of  time. 

(Clearly,  this  trick  only  works  once  for 
each  set  of  students,  since  the  effects  tend 
to  level  off  after  subsequent  administra- 
tions of  the  test). 
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THE  DANGERS  OF  CREAMING 

It  is  an  unfortunate  fact  of  adult  literacy 
that  programs  that  help  those  "hardest  to 
serve"  (for  example,  learners  who  are  both 
new  to  English  and  new  to  literacy)  have 
the  greatest  difficulties  showing  gains,  not 
only  because  their  learners  need  a great 
deal  of  time  until  progress  is  evident,  but 
because  the  kind  of  progress  they  are  mak- 
ing is  not  easily  captured  by  standardized 
multiple  choice,  paper  and  pencil  tests.  In 
addition,  programs  who  serve  these  stu- 
dents (often  community-based  organiza- 
tions) don't  have  the  resources  to  set  up 
testing  alternatives  appropriate  for  a low 
literacy  population. 

There  is  a danger,  then,  that  programs 
not  fully  committed  to  serving  learners 
who  need  both  special  support  and 
extended  time  will  decide  to  focus  their 
efforts  instead  on  those  students  who  most 
easily  advance,  since  the  incremental 
progress  of  "slower"  students  only  makes 
the  program  "look  bad." 

Thinking  along  those  lines,  ESL  pro- 
grams, for  example,  might  decide  to  focus 
the  curriculum  on  immigrants  with  higher 
levels  of  education,  rather  than  serving 
ESL  literacy  students. 

This  process  of  focusing  on  participants 
who  are  easy  to  serve  is  known  as  "cream- 
ing" and  has  long  been  decried  as  an  unin- 
tended outcome  of  programs  that  have 
signed  performance-based  contracts  (where 
funds  are  linked  to  learner  outcomes  and 
program  impacts,  such  as  job  placement). 

So  far,  not  many  public  debates  have 
taken  place  around  this  issue  in  adult  lit- 
eracy on  the  state  level,  but  concerns  are 
sure  to  arise  as  programs  realize  the  diffi- 
culties they  face  in  reporting  progress  across 
levels  in  the  time  periods  envisioned  by  the 
reporting  system. 
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SO  WHY  NOT  ASK 
FOR  AN  EXEMPTION? 

Two  solutions  to  the  problem  of  cream- 
ing seem  possible:  1)  set  aside  monies  so 
that  programs  can  develop  an  alternative 
assessment  for  lower  level  students  or 
2)  ask  that  learners  who  have  difficulty 
negotiating  paper  and  pencil  tests  be  ex- 
empted from  testing.  In  my  view,  exemp- 
tions, as  attractive  as  they  may  seems,  are 
not  the  best  solution  in  the  long  run,  since 
we  may  end  up  marginalizing  both  this 
group  and  programs  that  serve  them.  As 
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empt  from  accountability  requirements  is 
not  the  blessing  that  it  might  seem.  As  a 
rule,  if  certain  types  of  learners  are  excused 
from  testing,  they  tend  to  disappear  from 
the  radar  screen  of  administrators  and  are 
ignored  when  program  decisions  are  being 
made.  Furthermore,  it  is  difficult  to  ask  for 
funding  for  a population  for  whom  no  data 
is  available. 

I believe  that,  rather  than  asking  for  ex- 
emptions for  students  who  cannot  cope 
with  the  standardized  tests  approved  by  a 
state,  we  are  better  off  advocating  for  the 
development  of  an  alternative  assessment 
framework  for  this  group.  There  is  an 
additional  advantage  to  advocating  for  re- 
sources to  develop  an  alternative  assess- 
ment for  those  new  to  literacy.  Once  such 
an  assessment  is  developed  for  one  group, 
it  is  easier  to  acquire  the  resources  to  extend 
it  to  other  levels  and  other  populations. 

ALTERNATIVE  TESTING  FOR 
LOW  LITERATE  STUDENTS 

What  might  an  assessment  that  mea- 
sures the  incremental  changes  that  occur 
at  the  initial  levels  of  language  and  literacy 
development  look  like? 

It  is  entirely  possible  to  design  a frame- 
work that  allows  learners  to  demonstrate 
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If  a program 
wants  to  create 
an  assessment  that 
works  double  duty 
(as  a basis  for 
program 

improvement  and 
for  accountability), 
a further  step  is 
necessary:  the 
development  of 
scales,  rubrics,  and 
benchmarks  that 
indicate  the 
expectations  for 
any  given  level 
and  to  what  degree 
learners  are  close 
to  acquiring  the 
kind  of  knowledge, 
skills,  and 
strategies  that  are 
a core  part  of  our 
curriculum. 


what  they  can  say  and  understand  in  En- 
glish despite  limited  proficiency  (in  fact 
the  oral  interview  component  of  the  BEST 
test  does  just  that).  It  is  also  possible  to  de- 
sign a "can-do"  literacy  assessment  (of  the 
type  first  suggested  by  Lytle  and  Wolfe) 
based  on  the  kinds  of  texts  and  tasks  that 
those  new  to  literacy  deal  with  every  day. 
For  example,  tasks  could  be  designed  that 
allow  learners  to  select  pieces  of  print  that 
they  can  recognize  fairly  easily,  along  with 
those  that  give  them  some  difficulty  and 
others  that  pose  a still  greater  challenge 
(e.g.,  McDonald's  logos,  sale  signs,  50%  off 
promotions,  their  own  street  address,  a let- 
ter from  the  INS  or  the  TANF  office).  After 
selecting  these  print  pieces,  learners  would 
read  the  items  once  together  with  the 
friendly  teacher/  facilitator / assessor  and 
would  then  try  a few  text  pieces  that  they 
have  selected  on  their  own. 

The  assessor  rates  individual  perfor- 
mance on  a scale  without  making  a big 
deal  of  it.  On  the  third  round,  the  assessor 
might  select  an  item  that  is  slightly  more 
difficult  than  the  previous  one,  again  en- 
couraging the  person  to  discuss  the  item 
and  interpret  what  it  says.  Through  assess- 
ments of  this  sort,  we  should  be  able  to  tell 
to  what  extent  learners  can  handle  a vari- 
ety of  literacy  task  at  varying  levels  of  con- 
fidence and  proficiency.  It  would  help  us 
to  see  evidence  of  skills  worth  having,  such 
as:  1)  telling  an  electricity  bill  from  a phone 
bill  or  a notice  from  the  INS  from  a notice 
from  school,  2)  recognizing  certain  types 
of  applications  (housing,  employment;  citi- 
zenship), 3)  interpreting  real  life  environ- 
mental print  (reading  stop  or  danger 
signs),  or  4)  writing  a note  to  a repair  per- 
son, the  landlord,  or  the  worker  on  the 
next  shift. 

Asking  learners  to  select  tasks  that  they 


can  do  with  confidence  as  a starting  point 
for  assessment  and  then  moving  up  from 
there  is  not  limited  to  the  domains  of  prac- 
tical literacy.  For  those  interested  in  basic 
skills  acquisition  that  focus  on  the  subtasks 
of  reading,  one-on-one  student-initiated 
assessment  can  tell  us  to  what  extent  learn- 
ers have  developed  the  kind  of  "phonemic 
awareness"  that  allows  them  to  select  fa- 
miliar words  that  start  with  the  same  con- 
sonant or  identify  words  that  rhyme. 

Those  interested  in  basic  writing  profi- 
ciency can  ask  learners  to  select  an  evoca- 
tive photograph  or  some  other  prompt, 
discuss  it  with  the  facilitator  and  then 
write  the  response. 

Such  an  assessment  plus  conversation 
model  can  also  provide  baseline  data  on 
literacy  practices,  documenting  the  kind  of 
print  task  that  learners  engage  in  (looking 
at  TV  Guide;  reading  the  Bible;  checking 
the  horoscope  or  soccer  scores  (in  English 
or  in  a native  language  newspaper)  and  re- 
cording how  these  practices  change  over  time. 

Assessments  that  allow  learners  to  select 
a simple  task  and  then  branch  out  is  hardly 
a new  concept.  In  fact,  it  is  the  basis  for  the 
kind  of  "adaptive"  assessment  that  has 
been  used  in  computer-based  testing.  True 
this  this  type  of  assessment  requires  one- 
on-one  administration,  but  as  practices  in 
K-12  have  shown,  after  the  initial  intake  as- 
sessment has  been  completed,  teachers  can 
take  out  a few  minutes  with  each  student 
during  class  time  over  the  course  of  three 
weeks  or  so  to  document  what  learners  can 
do  that  they  could  not  do  before  (trained 
facilitators  could  do  short  "pull  out"  ses- 
sions as  well).  As  funding  for  adult  literacy 
is  increasing,  the  old  refrain  of  "there  is  no 
money  to  do  this"  no  longer  holds  true. 
There  are  alternatives  to  multiple  choice 
tests  and  we  must  advocate  for  their  devel- 
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opment  and  their  use  if  we  are  serious 
about  documenting  progress  for  all  learn- 
ers, including  those  who  still  struggle  with 
basic  literacy. 


what  participating  programs  have  deemed 
to  be  the  core  sets  of  knowledge,  skills,  and 
strategies  important  enough  to  teach  and 
test. 


BUILDING  AN  ASSESSMENT 
FRAMEWORK  THAT  YIELDS 
WORTHWHILE  RESULTS 


Developing  an  assessment  that  captures 
gains  at  the  lower  levels  is  only  the  starting 
point  in  a larger  effort  to  build  a system 
that  works. 

Other  efforts  are  needed,  at  both  the  lo- 
cal and  the  state  levels  so  that  we  don't 
end  up  with  an  accountability  system  that 
is  driven  in  large  part  by  what  current 
standardized  tests  are  able  to  measure.  If 
we  want  the  quality  of  adult  literacy  to  in- 
crease, we  need  an  approach  that  measures 
to  what  extent  learners  are  acquiring  the 
knowledge,  skills,  and  strategies  that  mat- 
ter in  the  long  rim.  These  might  include 
generative  skills,  such  as  gaining  meaning 
from  various  print  sources  important  to 
one's  life;  communicating  one's  thoughts 
and  ideas;  learning  how  to  learn;  knowing 
about  and  using  resources  effectively;  and 
learning  with  and  from  others  (along  with 
the  sub  skills  that  help  learners  become  in- 
creasingly more  proficient  in  these  areas). 

How  can  this  be  done?  At  the  local  level, 
a three-pronged  approach  might  be  neces- 
sary: 1)  finding  a way  to  live  with  the  cur- 
rently available  standardized  tests,  selecting 
the  "LOT"  — the  least  objectionable  test  — 
and  keeping  in  mind  the  principle  of  "first, 
do  no  harm"  to  students;  2)  convincing  the 
state  that  the  data  a program  has  provided 
over  the  years  are  at  least  as  valid  and  reli- 
able as  standardized  tests  such  as  the  TABE 
and  therefore  the  process  should  continue 
and  3)  work  with  others  to  develop  an  as- 
sessment system  that  reflects  the  realities 

of  adult  learners'  lives  and  focuses  on 
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COMPONENTS  OF  AN 
ALTERNATIVE  ASSESSMENT  SYSTEM 

Profiles  and  Portfolios 

What  might  be  the  components  of  such 
a system?  To  start  with,  any  program  con- 
cerned about  serving  different  groups  of 
learners  equally  well,  needs  to  collect  de- 
mographic information  that  captures  the 
kind  of  learner  characteristics  and  experi- 
ences that  may  have  a bearing  on  school 
success. 

After  all,  only  by  having  rich  descriptive 
information  can  we  know  what  learners 
want  and  need  to  do  with  English  and  lit- 
eracy (given  their  current  circumstances 
and  their  goals  for  the  future),  how  much 
schooling  they  have  had  (and  how  success- 
ful they  were),  and  what  the  print  and 
communication  challenges  are  that  they 
face  in  their  everyday  lives.  Having  de- 
scriptive information  of  this  kind  is  invalu- 
able since  it  allows  us  to  see  which  learners 
are  succeeding  in  our  programs  and  which 
are  languishing  (or  leaving)  because  their 
needs  are  not  met. 

This  information  can  be  collected  in  the 
form  of  profiles  that  travel  with  the  stu- 
dent and  to  which  teachers  and  learners 
contribute  on  an  ongoing  basis.  In  addition 
to  background  variables  such  as  age,  em- 
ployment status,  years  of  schooling,  coun- 
try of  origin  and  languages  spoken,  these 
profiles  can  1)  capture  current  literacy 
practices  (who  is  now  speaking  to  the  doc- 
tor without  a translator;  who  has  started  to 
pick  up  a newspaper  to  check  the 
weather);  2)  chart  shifts  in  learner  goals 
and  3)  record  changes  in  life  circumstances 
(new  job,  citizenship;  economic  self-suffi- 
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ciency)  important  to  stakeholders. 

In  these  profiles,  progress  can  be  cap- 
tured as  it  occurs  (requiring  only  a line  or 
two  for  two  or  three  students  per  class). 
Profiles  have  the  added  advantage  of  en- 
couraging teachers  to  create  opportunities 
for  learners  to  discuss  what  is  happening 
in  their  lives,  so  they  can  spend  some  time 
observing.  Profiles  of  this  sort  (also  known 
as  "running  records")  can  be  connected 
with  portfolios  that  demonstrate  student 
progress  through  writing  samples,  reading 
inventories,  and  various  types  of  perfor- 
mance tasks. 

If  a standardized  test  is  used,  results  can 
be  included  in  the  profile  as  well,  helping 
to  flesh  out  the  general  picture  of  achieve- 
ments and  struggles. 

FROM  LEARNER  SUCCESS 
TO  ACCOUNTABILITY 

This  must  be  said:  While  an  approach 
that  combines  rich  profiles  and  individual 
portfolios  will  produce  important  informa- 
tion on  individual  students  and  provide  in- 
sights into  the  relative  success  of  certain 
learner  groups,  it  does  not,  in  and  of  itself, 
yield  the  kind  of  data  needed  for  account- 
ability. After  all,  we  cannot  ship  boxes  of 
profile  folders  to  funders  to  have  them  re- 
alize what  a great  job  we  are  doing. 

To  make  profiles  work  for  funders,  a 
further  step  is  needed,  one  that  yields  data 
in  aggregate  form  so  that  policymakers  can 
get  a picture  of  the  shape  and  size  of  the 
forest,  not  just  a close-up  of  the  trees. 

To  measure  progress  and  report  to 
funders  who  is  getting  better  at  what,  pro- 
files need  to  include  the  following:  a broad 
set  of  language  and  literacy  tasks  that  are 
accompanied  by  rubrics,  scales,  and  bench- 
marks for  transition. 

Rubrics  are  used  to  indicate  what  expec- 


tations are  for  any  given  area  (face-to-face 
communication,  dealing  with  print,  access- 
ing resources  etc)  and  what  evidence  of 
success  might  look  like.  The  scales  that  ac- 
company the  rubrics  allow  us  to  document 
where  learners  fall  on  a continuum  of  pro- 
ficiency, documenting  what  they  can  do 
with  relative  ease,  where  they  succeed 
with  some  help,  and  where  they  are  strug- 
gling. 

Since  rubrics  and  scales  can  be  designed 
for  different  skill  domains  (SCANS  skills, 
communication  strategies,  navigating  sys- 
tems, civic  involvement,  learning  how  to 
learn,  empowerment,  etc)  and  for  various 
contexts  (school,  family,  community),  they 
can  easily  be  matched  to  the  goals  of  learn- 
ers and  adapted  to  the  focus  of  particular 
program.  They  also  allow  for  the  kind  of 
student  control  in  task  selection  discussed 
above. 

Once  rubrics  and  scales  are  in  place, 
meeting  accountability  requirements  that 
call  for  aggregate  data  becomes  relatively 
easy.  Since  the  descriptors  on  a scale  can 
easily  be  numbered  (from  1 for  "struggles" 
to  6 for  "no  problem",  say),  assessment  re- 
sults can  be  easily  compiled,  summarized, 
analyzed  and  reported  out.  If  matched 
with  demographic  profiles,  they  allow  a 
program  to  see  which  groups  of  learners 
are  being  served  well  by  the  program  and 
where  program  changes  are  in  order  be- 
cause success  is  lacking. 

The  beauty  is  that  this  kind  of  approach 
fulfills  the  same  function  as  standardized 
test:  learners  are  assessed  on  a variety  of 
skills  under  standard  conditions  with 
common  instruments  on  similar  tasks 
(yet  given  choices  in  task  selection  and  af- 
forded multiple  opportunities  to  shine  on 
tasks  that  matter  in  a given  context  impor- 
tant to  learners).  But  unlike  the  standard- 
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ized  tests  currently  available,  profile  as- 
sessments do  not  rely  on  multiple  choice, 
paper  and  pencil  items. 

Rather  they  give  learners  the  opportu- 
nity to  demonstrate  what  they  can  do  with 
language  and  literacy  through  more  open 
ended  assignments.  Furthermore,  profile 
approaches  to  assessment  can  be  adapted 
for  certain  learner  groups  and  modified  to 
match  the  focus  of  a particular  program 
(e.g.,  workplace,  family  literacy,  citizen- 
ship). Most  importantly,  perhaps,  they 
provide  rich  information  that  makes  sense 
to  teachers  and  learners,  information  that 
is  useful  to  programs,  not  just  funders. 

Why  then,  are  we  not  seeing  more  of 
these  kinds  of  assessments  ? While  ex- 
tremely worthwhile  and  high  in  validity, 
these  types  of  assessment  carry  a signifi- 
cant burden:  they  require  consensus  build- 
ing on  what  is  worth  teaching  and  learning 
and  a common  understanding  of  what  evi- 
dence of  success  might  look  like  for  any 
given  skill  domain.  To  be  successful,  pro- 
files and  portfolios  have  to  be  integrated 
into  the  curriculum  and  ongoing  assess- 
ment must  either  be  part  of  the  day-to-day 
teaching  we  do,  or  time  must  be  set  aside 
at  intake  to  establish  baseline  and  toward 
the  end  of  a teaching  cycle  to  document 
progress.  If  that  means  the  end  of  open-en- 
try/open  exit  as  we  know  it  and  forces  us 
into  shorter  instructional  cycles  that  have  a 
clear  teaching/learning  focus,  so  be  it.  To 
give  such  a framework  a chance,  a signifi- 
cant amount  of  teacher  orientation,  train- 
ing and  buy-in  will  be  needed. 

Clearly,  there  are  not  many  adult  lit- 
eracy programs  that  have  the  commitment, 
energy  and  resources  to  embark  on  that 
endeavor,  although  some,  like  the  Arling- 
ton Education  and  Employment  Program 
in  Virginia  are  well  on  their  way.  But, 


given  sufficient  advocacy  from  local  pro- 
grams along  with  a modicum  of  political 
will  on  the  part  of  state  directors  and  other 
funders,  teams,  working  groups  and  con- 
sortia could  be  set  up  to  develop  an  assess- 
ment framework  that,  if  not  based  on 
profiles,  at  least  includes  them.  In  fact,  the 
National  Institute  for  Literacy,  is  moving 
in  that  direction,  developing  an  assessment 
framework  that  combines  the  use  of  alter- 
native assessments  with  standardized  tests 
where  appropriate  in  order  to  capture  the 
gains  that  learners  make  who  are  part  of 
the  "Equipped  for  the  Future”  initiative. 

What  then  is  the  bottom  line,  given  the 
current  climate  of  accountability  for 
accountability's  sake?  We  have  several  op- 
tions: we  can  decide  that  cynicism  is  the 
only  sane  response  to  the  current  require- 
ments, live  with  standardized  tests  as  best 
as  we  can,  try  to  lay  low,  figuring  "this  too 
shall  pass,"  or  commit  ourselves  to  fight- 
ing for  a saner  system  for  our  own  sake 
and  that  of  our  students.  On  the  local  level, 
we  must  be  prepared  to  work  with  others 
to  decide  on  the  focus  of  our  programs  and 
be  willing  to  map  out  a core  set  of  knowl- 
edge, skills  and  strategies  that  matter. 

At  the  federal  level,  we  must  push  for 
an  accountability  system  that  is  driven  not 
by  what  the  current  standardized  tests  are 
able  to  assess  (which  is  rather  limited),  but 
by  outcomes  that  reflect  what  sound  adult 
literacy  programs  should  be  all  about.  Fur- 
thermore, if  we  are  asked  to  show  account- 
ability related  to  outcomes  and  impacts, 
we  must  be  given  the  resources  to  docu- 
ment success  in  meaningful  ways. 

Finally,  while  we  may  need  to  play  the 
accountability  game  for  the  time  being,  we 
can  also  work  toward  a system  that  mea- 
sures effectiveness  where  it  counts:  adult 
learners  acquiring  the  kinds  of  knowledge. 
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skills  and  strategies  that  are  important  to  If  we  give  up  too  soon,  we  will  only 
them  now  and  that  matter  in  the  long  run.  marginalize  adult  literacy  further. 
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Tips  on  Conferencing 


conferencing  is  a one-on-one  meet- 
ing between  the  learner  and  the 
teacher.  If  conducted  at  the  begin- 
ning of  a cycle,  conferencing  can  enable  the 
learner  to  privately  share  with  the  teacher 
his/her  hopes  and  fears  about  the  class 
and  any  special  considerations  s/he  may 
have,  as  well  as  to  help  the  learner  develop 
goals  for  the  class.  Conferencing  at  this 
stage  is  also  a good  way  to  establish  a 
supportive,  open  relationship  between  the 
learner  and  teacher.  When  used  mid-way 
and  at  the  end  of  a cycle  or  semester, 
conferencing  is  a way  to  help  the  learner 
see  his/her  progress,  to  review  and  revise 
the  learner's  goals,  to  help  the  teacher  get 
feedback  for  classroom  and  program  im- 
provement, and  to  plan  for  next  steps. 

POINTS  FOR  THE  TEACHER 

• Conference  with  learners  three  times 
during  the  cycle  or  semester:  at  start-up, 
mid- way,  and  at  the  end. 

• Keep  the  conference  to  30-45  minutes. 
Shorter  than  that  may  be  too  superficial; 
longer  may  limit  time  to  meet  with 
other  learners. 

• Talk  about  the  purpose  of  conferencing 
with  your  learners  during  class  time.  Ex- 
plain what  conferencing  sessions  are, 
their  purpose,  and  how  much  time  they 
tend  to  take.  Explain  that  the  conference 
is  a two-way  conversation  designed  to 
inform  both  the  learner  and  the  teacher 
on  ways  to  continue  to  grow  and  im- 
prove. Discuss  the  kinds  of  questions 
vou  ask  and  ask  learners  if  they  have  . 


any  suggestions  to  make  the  conference 
a valuable  one. 

• Conferencing  done  at  the  start-up  of  a 
cycle  may  best  be  conducted  outside 
class,  to  make  a special  connection  with 
each  person,  to  really  hear  what  s/he 
has  to  say,  to  answer  questions,  and  to 
help  them  determine  what  they  need 
and  want  to  do. 

• Incorporate  mid-way  and  end-of-semes- 
ter  conferencing  sessions  during  regular 
class  time  so  that  neither  of  your  sched- 
ules are  overburdened.  Schedule  them 
ahead  of  time  so  that  learners  are  ex- 
pecting them,  and  conduct  them  when 
the  class  is  engaged  in  a learning  activ- 
ity when  your  presence  is  not  needed 
(when  they  are  writing  in  their  journals, 
quietly  reading,  working  in  groups  on  a 
project,  etc.). 

• For  mid-way  and  end-of-semester  con- 
ferences, you  and  the  learner  can  pre- 
pare by  having  materials  ready  which 
could  show  learners'  progress  (portfo- 
lios that  should  include  writing  and 
reading  that  is  important  to  the  learner, 
initial  goal-setting  lists,  learning  con- 
tracts, etc.).  Let  the  learner  know  before- 
hand the  kinds  of  questions  you  will  be 
asking  (such  as  ways  the  class  could  be 
improved).  Also  consider  any  key  points 
or  critical  events  that  you  want  to  make 
sure  you  have  time  to  discuss  with  the 
learner. 

• If  you  find  that  you  really  do  not  have 
the  time  to  meet  individually  with  leam- 
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ers  even  during  class  time,  consider  us- 
ing dialogue  journals.  The  journal  can  be 
adapted  for  assessment  purposes  by  en- 
gaging the  learner  in  written  conversa- 
tions about  their  goals  and  how  their 


goals  may  have  changed,  their  thoughts 
about  how  they  are  meeting  their  goals, 
progress  that  you  see,  ideas  about  how 
the  class  or  the  program  could  be  im- 
proved, etc. 
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An  Action  Research  Project 

Authentic  and  Learner-Centered  Assessment 
in  the  Beginning  ESOL  Classroom 

reprinted  from  Building  Together:  The  Inquiry  Writings,  The  North  Carolina  Adult  ESOL 
Curriculum  Frameworks  Inquiry  Project,  Literacy  South,  1998,  (919)  682-8108 


Underlying  the  ideas  about  assess- 
ment presented  in  this  paper  is  an 
assumption  about  the  purpose  of 
ESOL  education  which  may  seem  obvious 
but  which  I nevertheless  feel  is  worth  stat- 
ing since  the  practices  of  many  ESOL  pro- 
grams and  practitioners  (including  myself) 
are  not  always  consistent  with  it.  This  as- 
sumption is  that  the  ultimate  purpose  of 
ESOL  learning  is  to  help  learners  effec- 
tively use  the  English  language  to  commu- 
nicate for  purposes  that  they  deem  to  be 
meaningful  and  useful  to  themselves.  Ac- 
ceptance of  this  principle  necessarily  leads 
to  the  question  of  how  assessment  of  ESOL 
learning  can  and  should  be  linked  to  the 
authentic  purposes  ESOL  learners  have  for 
learning  English.  I believe  that  truly  useful 
assessment  in  the  ESOL  classroom  must  at 
the  very  least  help  both  learner  and  teacher 
to  identify  the  learner's  authentic  purposes 
for  learning  English,  to  gain  a clear  and  on- 
going perspective  on  how  the  learner  is 
progressing  towards  his/her  purposes  for 
learning  English,  and  to  evaluate  how  well 
learning  activities  used  in  the  classroom 
are  assisting  this  progress. 

For  our  assessment  strategies  to  accom- 
plish these  ends,  they  must  necessarily  be 
both  authentic  and  learner-centered.  By 
"authentic"  assessment  I mean  assessment 
based  on  observation  of  how  well  the 
learner  is  actually  able  to  use  English  for 
genuine  communication  for  real-life  pur- 
poses. Learner-centered  assessment  I will 
define  here  as  assessment  of  learning  that 
is  either  implemented  by  the  learners 
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themselves  or,  if  implemented  with  the 
assistance  of  the  teacher,  is  nevertheless 
clearly  understood  by  the  learners  to  be  a 
means  for  understanding  their  own  learn- 
ing process,  for  evaluating  their  progress, 
and  for  evaluating  the  usefulness  of  the 
cumculuxn  tltcy  circ  pcirhcip  citing  m.  uuw 
co-directing  with  the  teacher. 

Assessment  that  is  both  authentic  and 
learner-centered  is  vital  to  good  ESOL 
teaching/learning.  Such  assessment  en- 
courages learners  to  think  about  and  to 
take  responsibility  for  their  own  learning 
and  is  essential  to  producing  curriculum 
that  is  suited  to  the  particular  needs  and 
goals  of  each  learner.  Also,  it  helps  learn- 
ers not  only  to  become  cognizant  of  the 
fact  that  they  are  learning,  if  indeed  they 
are,  (this  is  particularly  important  in  those 
cases  when  learners  may  have  certain  ex- 
pectations about  what  language  learning  is 
supposed  to  look  like  which  they  do  not 
see  realized  in  their  ESOL  class)  but  to  be 
aware  of  how  they  are  moving  closer, 
through  the  steps  of  interrelated  learning 
activities,  to  fulfilling  their  own  intrinsi- 
cally-motivating purposes  for  learning  En- 
glish. Could  the  absence  of  this  kind  of 
assessment  be  one  of  the  reasons  for  that 
phenomena  which  I am  sure  many  adult 
ESOL  instructors  experience,  namely  wit- 
nessing a quiet  student  enter  into  one's 
ESOL  program  and  classroom,  come  to 
class  a couple  times  and  never  return?  A 
further  implication  of  learner-centered  as- 
sessment, if  it  is  to  make  learners  continu- 
ally cognizant  of  how  they  are  learning,  is 
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that  such  assessment  needs  to  be  ongoing 
and  integrated  into  daily  classroom  activ- 
ity. 

Having  taught  beginning  level  ESOL  to 
adult  learners  in  the  North  Carolina  Com- 
munity College  system  for  several  years,  I 
have  become  increasingly  aware  of  the  im- 
portance of  providing  authentic  and 
learner-centered  assessment  for  my  stu- 
dents for  the  reasons  stated  above.  I have 
also  become  aware  of  the  obstacles  to 
implementing  such  assessment  that  ESOL 
teachers,  and  especially  beginning  level 
ESOL  teachers,  face  in  a community  col- 
lege setting.  Therefore  I chose  as  the  goal 
of  my  action  research  project  to  explore 
ways  for  accomplishing  authentic  and 
learner-centered  assessment  within  begin- 
ning level  adult  ESOL  classrooms  in  a 
North  Carolina  Community  College  set- 
ting. How  can  this  be  done  despite  the 
obstacles  presented  by  the  very  limited 
ability  of  many  learners  at  this  level  to  un- 
derstand and  use  English  and  despite  the 
open-entry/ open-exit  and  attendance  poli- 
cies of  North  Carolina  Community  College 
system?  These  policies,  in  particular,  create 
a challenging  situation  for  beginning  level 
ESOL  teachers  who,  because  of  these  poli- 
cies, must  deal  with  having  new  students 
with  practically  no  English  language  profi- 
ciency entering  their  classes  at  any  time 
during  a term  while  other  learners  in  these 
classes  may  already  have  made  substantial 
progress.  Linder  these  conditions,  imple- 
menting authentic  and  learner-centered  as- 
sessment becomes  enormously  challenging. 
A certain  degree  of  English  language  profi- 
ciency on  the  part  of  learners  would  seem 
to  be  necessary  in  order  for  learners  to  par- 
ticipate in  and  understand  the  assessment. 
Constant  fluctuation  in  the  student  popula- 
tion makes  it  very  difficult  for  the  teacher 
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to  be  consistent  in  the  manner  in  which 
he/ she  assesses  all  of  learners  attending 
his/her  class.  To  help  answer  my  question, 
I decided  I would: 

1)  research  current  literature  regarding  au- 
thentic and  learner-centered  assessment 
for  ESOL  learners, 

2)  design  and  test  in  my  classroom  assess- 
ment activity  for  beginning  level  ESOL 
learners  which  is  at  the  same  time  a 
learning  activity, 

3)  draw  conclusions  from  my  research  and 
my  classroom  experience  about  how  au- 
thentic and  learner-centered  assessment 
may  be  implemented  in  the  beginning 
level  ESOL  classroom  in  an  ongoing  and 
systematic  manner. 


This  paper  documents  the  results  of  my 
project.  I will  first  discuss  some  of  the  pur- 
poses for  authentic  and  learner-centered 
assessment  in  ESOL  education  and  some 
specific  assessment  tools  I learned  about 
through  my  research.  Then  I will  talk 
about  an  assessment/leaming  activity 
which  I designed  to  help  my  own  students 
reflect  on  their  purposes  for  learning  En- 
glish. I tested  this  in  my  classroom  in  April 
of  1998. 1 will  discuss  how  this  activity 
worked  with  my  students.  Finally,  I will 
draw  conclusions  about  the  implementa- 
tion of  authentic  and  learner-centered  as- 
sessment in  beginning  level  ESOL 
classrooms  within  North  Carolina  Com- 
munity Colleges  based  on  my  research  and 
classroom  experience. 
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WHAT  I LEARNED  FROM  MY  READINGS 
ON  ASSESSMENT 

Some  Purposes  for  Assessment: 

Through  my  research  of  current  litera- 
ture on  assessment  in  ABE  and  ESOL,  I 
identified  several  different  specific  pur- 
poses for  assessment  that  require  the  use 
of  different  kinds  of  assessment  tools  in 
ESOL  classrooms.  Among  these  purposes 
for  assessment  are  the  following  which  I 
believe  any  comprehensive  assessment 
strategy  should  seek  to  accomplish: 

• to  determine  learners r resl-life  purposes 
for  learning  English 

• to  assess  learners'  language  acquisition 
(i.e.,  both  the  language  skills  learners 
possesses  as  well  as  language  abilities 
they  still  need  to  leam  to  achieve  their 
purposes) 

• to  assess  the  degree  to  which  learners 
are  able  to  effectively  use  English  to 
communicate  for  authentic  purposes 
both  inside  and  outside  of  the  classroom 

• to  encourage  learners  to  reflect  on  and 
take  responsibility  for  their  own  learn- 
ing 

• to  give  both  learners  and  teachers  a 
clear  sense  of  learners'  progress  towards 
learners'  purposes  for  learning  English, 
to  inform  and  guide  learner-centered 
curriculum  development,  to  assess 
learning  styles  and  learning  strategies 
of  learners  in  order  to  create  learning 
activities  that  are  more  useful  to  each 
learner 

• to  evaluate  learning  activities  used  in 
the  classroom  in  terms  of  their  effective- 
ness in  promoting  learner  progress 

All  of  these  purposes  for  assessment  are 
mutually  supportive  and  important  to  op- 


timizing English  language  learning  for  all 
learners  that  enter  our  ESOL  programs. 
These  purposes  also  all  require  authentic 
and/ or  learner-centered  approaches  to  be 
done  effectively.  I chose  to  focus  particu- 
larly on  the  first  of  these  purposes,  for  the 
part  of  my  project  involving  inquiry  into 
my  own  classroom,  because  I see  the  pro- 
cess of  helping  learners  to  reflect  on  and 
determine  what  their  purposes  are  for 
learning  English  as  the  first  essential  step 
to  be  taken  when  attempting  to  implement 
an  authentic,  learner-centered  assessment 
strategy.  I will  describe  the  results  of  the 
research  I accomplished  in  my  own  class- 
room later  in  this  paper,  but  first  will  dis- 
cuss some  other  authentic  and/ or  learner- 
centered  assessment  tools  that  I learned  of 
in  my  survey  of  literature  on  this  topic. 

Some  Authentic  and/or  Learner-centered 
Assessment  Tools: 

Below,  I describe  some  of  the  authentic 
and/ or  learner-centered  assessment  tools 
which  have  been  used  in  learner-centered 
ESOL  and/ or  ABE  classrooms  which  I 
learned  of  through  my  readings.  I have 
included  my  reflections  on  some  of  these 
with  particular  regard  to  how  feasible  I 
believe  their  implementation  would  be  in 
classrooms  like  mine.  I have  categorized 
these  tools  into  four  groupings: 

1)  Assessment  of  Learner  Purposes  for 
Learning  English, 

2)  Assessment  of  Learner  Progress  towards 
Self-defined  Purposes  for  Learning  En- 
glish and  of  English  Language  Acquisi- 
tion in  General, 

3)  Assessment  of  Learning  Styles:  Promot- 
ing Reflection  on  How  Each  Learner 
Learns  Best,  and 

4)  Assessment  of  Learning  Activities. 


I see  the  process 
of  helping  learners 
to  reflect  on  and 
determine  what 
their  purposes 
are  for  learning 
English  as  the  first 
essential  step  to 
be  taken  when 
attempting  to 
implement  an 
authentic,  learner- 
centered  assess- 
ment strategy. 
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For  a goal  setting- 
activity  to  be  truly 
useful,  especially 
with  beginning 
level  ESOL  learn- 
ers, it  seems  to  me 
that  it  needs  to  be 
integrated  into  a 
learning  activity 
which  assists  learn- 
ers to  understand 
the  questions  being 
asked  of  them  and 
which  guides  them 
to  reflect  on  and 
discover  what  their 
authentic  needs / 
goals  are. 


Some  of  the  tools  can  fit  into  more  than 
one  of  these  categories  but  I have  only 
listed  them  once  to  avoid  repetition.  Most 
of  these  tools  can  be  implemented  as  class- 
room activities  but  some  require  documen- 
tation to  be  done  outside  of  class  time. 

1)  Assessment  of  Learner  Purposes  for 
Learning  English: 

• Teacher  presents  a picture  or  photo- 
graph which  shows  a social  situation 
that  he/ she  has  reason  to  believe  may 
be  problematic  for  some  of  the  learners 
(i.e.  a "code").  Teacher  then  promotes 
and  takes  note  of  ensuing  discussion  to 
determine  issues  of  importance  to  learn- 
ers and  to  design  future  curriculum 
more  closely  suited  to  learner  needs. 

• Teacher  shows  learners  a variety  of  pic- 
tures and/ or  photographs  of  situations 
in  which  communication  is  occurring  in 
American  society.  Learners  choose  from 
among  these  pictures  those  which  repre- 
sent situations  in  which  they  feel  it  is 
most  important  for  them  to  use  English. 
As  an  alternative,  learners  may  draw 
their  own  pictures  of  situations  in  which 
they  feel  it  is  most  important  for  them  to 
be  able  to  use  English.  This  information 
is  used  to  determine  learner  purposes 
for  using  English  and  to  inform  curricu- 
lum development. 

• Students  fill  out  a questionnaire  regard- 
ing why  they  want  to  learn  English  and 
what  their  goals  are. 

Reflections:  This  last  activity  is  similar  to 
a method  for  determining  student  goals  al- 
ready used  at  Wake  Technical  Community 
College.  Wake  Tech  uses  a check  list  with 
predetermined  goals  for  ESOL  learners  to 
choose  from  — goals  such  as  "Higher 
Level  of  Independent  Living,"  "Increase 
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Daily  Living  Skills,"  "Improve  Reading 
Skills /Comprehension,"  "Improve  Com- 
munication Skills"  etc.  The  problem  with 
this  method  is  that  it  rarely  gives  much  in- 
formation to  the  teacher  that  could  be  use- 
ful in  curriculum  development.  Learner 
responses  to  such  surveys  are  always  too 
general  to  be  helpful  in  this  regard.  Even 
when  learners  are  asked  through  open- 
ended  questions  on  a goal  setting  ques- 
tionnaire to  identify  their  own  goals, 
learners  still  tend  to  give  vague  responses 
such  as  "to  learn  more  English"  unless 
they  receive  additional  guidance  and  sup- 
port to  reflect  more  critically  about  their 
needs  and  goals.  This  is  especially  true  at 
beginning  levels.  For  a goal  setting-activity 
to  be  truly  useful,  especially  with  begin- 
ning level  ESOL  learners,  it  seems  to  me  . 
that  it  needs  to  be  integrated  into  a learn- 
ing activity  which  assists  learners  to  un- 
derstand the  questions  being  asked  of 
them  and  which  guides  them  to  reflect  on 
and  discover  what  their  authentic  needs/ 
goals  are.  Also,  I believe  it  would  be  help- 
ful to  teachers  and  learners  to  define  these 
learner  goals  in  terms  of  what  I would  call 
communication  targets.  A communication 
target  would  identify  a specific  kind  of 
message  the  learner  wants  to  be  able  to  un- 
derstand and/or  effectively  communicate 
as  well  as  the  social  context  in  which  the 
learner  needs  or  wants  to  receive  and/ or 
convey  this  message. 

2)  Assessment  of  Learner  Progress  towards 
Self-defined  Purposes  for  Learning 
English  and  of  English  Language  Acqui- 
sition in  General: 

• Learners  orally  (or  through  non-verbal 
means)  indicate  to  the  teacher  when 
they  don't  understand  something  or  feel 
they  need  more  help  in  certain  areas. 
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Reflections : This  method  of  learner-cen- 
tered assessment  occurs  naturally  in  ESOL 
classrooms  where  learners  feel  comfortable 
and  where  dialogue  among  learners  and 
between  learners  and  teacher  is  accepted 
and  encouraged.  Indeed,  it  is  such  a natu- 
ral occurrence  that  it  is  often  taken  for 
granted  by  teachers.  I,  for  one,  did  not 
even  think  of  it  as  a form  of  assessment  un- 
til after  the  research  and  reflection  on  the 
subject  of  assessment  that  my  project  in- 
volved. 

• Learners  periodically  (daily  or  weekly) 
fill  out  a self-assessment  form  indicating 
new  words,  phrases,  and  language 
structures  they  have  learned  and/ or 
used  inside  and  outside  of  class.  This 
form  should  also  ask  learners  to  relate 
their  progress  in  language  acquisition  to 
their  communication  targets  and  should 
ask  learners  to  identify  areas  in  which 
they  believe  they  need  more  assistance. 

Reflections:  In  a recent  article  about  one 
ESOL  program's  ongoing  assessment  strat- 
egy, authors  K.  Ebbit,  P.  Lee,  P.  Nelson, 
and  J.  Wheeler  wrote  of  a three  part  assess- 
ment strategy  being  implemented  at  the 
Community  Learning  Center  in  Cambridge 
and  Somerville,  MA.  The  three  parts  of 
their  assessment  strategy  are  1)  goal-set- 
ting  exercises,  2)  weekly  written  self-as- 
sessment on  which  learners  indicate  what 
new  language  abilities  they  believe  they 
have  learned  using  standardized  forms  de- 
veloped for  this  purpose,  and  3)  a progress 
record  which  consists  of  a checklist  filled 
out  by  students  at  the  end  of  a unit  and 
kept  as  a record  to  help  learners  recall 
what  topics  and  language  skills  were  cov- 
ered in  their  class  over  a given  period.  This 
provides  learners  an  opportunity  to  indi- 
cate whether  they  feel  they  have  mastered 


a topic  or  skill  to  their  satisfaction  or 
whether  they  feel  they  would  like  to  learn 
more  about  or  need  more  practice  in  a cer- 
tain area.  Judging  from  the  experiences 
which  the  authors  describe,  it  would  seem 
(as  I certainly  would  have  suspected  given 
my  own  experience  teaching  beginning 
level  ESOL)  that  the  second  part  of  this 
approach  to  assessment  (i.e.  the  weekly 
written  self-assessment)  was  the  most 
problematic  of  the  three,  especially  with 
low  level  ESOL  learners,  due  to  the  exces- 
sive amount  of  time  needed  to  help  these 
learners  understand  the  forms  (both  the 
language  and  the  purpose  of  the  form), 
and  fill  them  out.  It  also  seemed  to  me  that 
some  of  the  information  asked  for,  such  as 
"What  new  words  have  you  learned?", 
was  not  a particularly  useful  assessment  of 
ESOL  learning.  Would  it  not  be  more  use- 
ful to  teachers  and  learners  to  also  know 
how  and  in  what  contexts  these  new  words 
were  used  and  whether  or  not  the  learner 
was  able  to  communicate  effectively  using 
these  words  (i.e..  Communicative  effective- 
ness is  an  outcome  of  more  factors  than 
simply  the  knowledge  of  vocabulary. 

Other  factors  that  determine  a person's 
ability  to  effectively  communicate  are  pro- 
nunciation, understanding  and  use  of  ap- 
propriate language  structures,  understand- 
ing of  cultural  norms,  self-confidence/ 
assertiveness  etc.  and  need  also  to  be  as- 
sessed to  accurately  evaluate  the  learner's 
ability  to  communicate  effectively  in  En- 
glish). 

• Learners  are  periodically  given  formal 
oral  and  written  exams  which  reflect 
themes  and  language  taught  in  class 
over  a given  period  of  time.  On  a more 
limited  scale,  any  cloze  exercise  given 
after  introducing  new  vocabulary  and 
language  structures  can  be  viewed  as  a 
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mini-exam  for  determining  how  much 
of  the  information  taught  in  the  form  of 
vocabulary,  grammar  and/ or  spelling 
was  retained  by  the  learners. 

Reflections:  It  seems  to  me  that  this 
method  could  help  to  reinforce  what  learn- 
ers have  studied  in  previous  classes  and 
give  both  teachers  and  learners  a better 
sense  of  what  areas  learners  may  need 
more  practice  in.  For  tests  to  do  this,  how- 
ever, they  would  need  to  be  prepared  by 
teachers  (as  opposed  to  standardized)  and 
be  directly  related  to  the  material  learners 
have  covered  before  the  test  is  given  (un- 
like the  B.E.S.T.  test).  My  guess  is  that  it 
would  be  challenging  to  implement  this  as- 
sessment strategy  other  than  in  the  simple 
form  of  cloze  exercises  at  the  beginning 
level  because  of  limited  language  profi- 
ciency and  literacy  skills  of  many  begin- 
ning ESOL  students. 

• LEA  (Language  Experience  Approach) 
learning  activities  can  also  be  viewed  as 
assessment  activities  since  they  record 
learners'  knowledge  of  vocabulary  and 
grammar  at  a given  moment.  [For  more 
information  on  the  Language  Experi- 
ence Approach  see  Margaret  Gransee's 
paper  in  this  collection.] 

• Using  a teacher's  observation  log  which 
may  be  in  the  form  of  a journal  or  a stan- 
dardized form,  the  teacher  takes  a little 
time  at  the  end  of  each  class  to  take  note 
of  learners'  abilities  and  needs  which 
learners  demonstrated  during  class.  This 
method  may  be  used  to  assess  the  fol- 
lowing: 

1)  learners'  ability  to  communicate  orally 
(by  noting  learner  miscues  and  newly 
demonstrated  abilities), 

2)  learners'  literacy  skills  (also  by  noting 


learners'  miscues  and  newly  demon- 
strated abilities  as  evidences  in  samples 
of  their  writing  or  in  their  demonstrated 
reading  comprehension), 

3)  the  degree  to  which  learners  seem  "en- 
gaged" by  the  learning  activities  (in  or- 
der to  evaluate  learners'  interest  in  a 
given  learning  activity  and  the  effective- 
ness of  the  activity  for  each  learner), 

4)  learners'  self-confidence,  and 

5)  the  interests /survival  needs  that  learn- 
ers have  in  their  everyday  lives  as  evi- 
denced by  comments  made  in  casual 
conversation  with  them. 

Reflections:  This  assessment  tool  would 
probably  be  useful  to  the  teacher  (by  help- 
ing him/her  to  take  note  of  each  learner's 
demonstratedlanguage  skills,  language 
needs,  and  learning  styles  and  to  adapt 
curriculum  to  these)  and  would  be  a valu- 
able guide  to  future  curriculum  develop- 
ment, but  it  wouldn't  give  the  learner  an 
indication  of  the  progress  he/she  is  mak- 
ing (unless  it  is  regularly  shared  with  the 
learners).  It  also  could  become  very  time 
consuming  for  teachers  to  consistently  do 
this  kind  of  assessment  after  each  class  for 
each  student  who  attended  the  class.  To 
expect  teachers  who  are  not  paid  for 
preparation  time  to  find  more  time  outside 
of  their  paid  teaching  time  to  consistently 
keep  track  of  this  kind  of  data  is,  it  seems 
to  me,  a lot  to  ask.  On  the  other  hand,  per- 
haps if  the  teacher's  log  had  a simple  for- 
mat and  if  the  teacher /student  ratio  were 
ideal  and  attendance  were  consistent,  then 
it  would  be  a more  feasible,  and  no  doubt 
valuable,  assessment  tool  for  beginning 
adult  ESOL  teachers  in  NC  Community 
Colleges. 
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• The  teacher  uses  a tape  recorder  to  peri- 
odically assess  each  learner's  ability  to 
orally  communicate  in  English.  The 
teacher  develops  a standard  set  of  ques- 
tions to  ask  learners  at  different  times  in 
the  course  of  a semester  and  the  result- 
ing interviews  are  taped.  Both  teacher 
and  learner  can  thus  assess  learner 
progress  in  English  language  acquisition 
over  a period  of  time  by  listening  to 
earlier  and  then  later  tape  recordings  of 
interviews  with  a given  learner  and  not- 
ing the  change  in  the  learner's  ability  to 
communicate  effectively  in  response  to 
the  same  questions. 

Reflections:  While  this  method  sounds 
like  it  would  be  useful,  it  doesn't  seem  to 
lend  itself  to  use  in  the  beginning  level 
ESOL  classes  I have  taught  at  Wake  Tech 
and  other  community  colleges.  The  prob- 
lem is  that  I can't  afford  to  take  time  to  do 
this  with  each  student.  The  inconsistent  at- 
tendance of  learners  further  complicates 
things  since  it  would  require  me,  if  I felt 
obligated  to  assess  every  student  in  like 
manner,  to  interrupt  my  planned  learning 
activity  whenever  a learner  that  needed  to 
be  assessed  happened  to  show  up  in  class 
in  order  to  take  time  to  interview  and  tape 
him/her.  This  method  might  be  more  vi- 
able if  I had  a regular  teacher's  aide  to  help 
(the  one  intake  person  at  most  of  Wake 
Tech's  larger  off-campus  adult  learning 
sites  might  be  able  to  help  with  this  but 
they  seem  to  me  to  already  be  over- 
worked). 

3)  Assessment  of  Learning  Styles:  Promot- 
ing Reflection  on  How  Each  Learner 
Learns  Best: 

• Group  Activity:  The  class  is  divided  into 
two  groups.  One  group  is  asked  to  com- 
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plete  the  phrase  "It's  easy  to  learn 
when...."  The  other  group  is  asked  to 
complete  the  phrase  "It's  hard  to  learn 
when...."  Ideas  which  come  up  in  each 
group's  discussion  are  then  shared  with 
the  whole  class.  Teacher  promotes  dis- 
cussion of  the  responses  each  small 
group  generated.  ( Adventures  in  Assess- 
ment, Vol.  4) 

Reflections:  This  activity  may  work  well 
to  encourage  learners  to  begin  to  think 
about  their  own  learning,  about  how  they 
learn  best,  and  also  give  the  teacher  infor- 
mation about  students'  learning  styles  as 
well  as  feedback  regarding  which  kinds 
of  learning  activities  are  most  useful  for 
which  students.  However,  this  activity 
would  require  of  beginning  ESOL  learners 
a degree  of  proficiency  in  English  that  they 
probably  would  not  acquire  until  they  are 
ready  to  move  to  a higher  level  of  ESOL 
learning. 

4)  Assessment  of  Learning  Activities: 

• Learners  respond  to  a questionnaire  re- 
garding which  of  the  learning  activities 
in  their  class  are  working  and  which  are 
not  working  for  them.  This  question- 
naire would  be  filled  out  regularly 

(i.e.  daily  or  weekly)  by  learners. 

• As  a regular  class  ritual,  the  teacher  takes 
a period  of  time  at  the  end  of  each  class 
to  ask  learners  what  they  learned  today, 
which  learning  activities  were  most  use- 
ful for  them,  what  they  liked  and  dis- 
liked about  the  class,  what  else  they 
want  to  learn,  etc.  The  teacher  takes  note 
of  learner  responses.  Once  taking  time 
to  get  this  kind  of  feedback  from  learn- 
ers becomes  an  established  routine 
"learners  will  feel  more  comfortable 
with  giving  feedback  and  move  from 
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'everything  is  fine'  to  'I  would  like  to  see 

[blank]  in  class.'"  (Gear,  p.  47) 

MY  CLASSROOM  EXPERIENCE: 

ONE  ASSESSMENT/LEARNING  ACTIVITY 
FOR  DETERMINING  LEARNER 
PURPOSES  FOR  LEARNING  ENGLISH 

I was  teaching  a Beginning  level  ESOL 
class  and  was  about  halfway  through  the 
spring  term  of  classes  at  Wake  Technical 
Community  College  when  I decided  to  see 
if  my  learners  had  acquired  enough  En- 
glish to  be  able  to  engage  in  some  mean- 
ingful dialogue  about  what  was  important 
to  them  in  their  lives  and  to  what  purposes 
for  communicating  in  English  they  gave 
highest  priority.  I designed  an  assessment/ 
learning  activity  which  I entitled  "What  is 
important?"  On  the  night  I tried  out  this 
activity,  I had  nine  students  in  attendance. 
Of  these  nine,  seven  were  regulars,  one 
was  a student  who  sporadically  attended, 
and  one  had  come  to  my  class  for  the  first 
time.  Seven  of  the  nine  were  from  Mexico, 
one  was  from  Iran,  and  one  from  Vietnam. 

I began  the  activity  by  writing  the  ques- 
tion on  my  overhead  projector,  "What  is 
important?"  After  spending  some  time  ex- 
plaining the  meaning  of  "important"  (for 
the  sake  of  one  student  who  didn't  under- 
stand the  word),  I drew  a picture  of  a plant 
and  asked  the  class  to  identify  what  it  was. 
Then  I drew  a sun  shining  on  the  plant  and 
a rain  cloud  raining  on  the  plant  and  asked 
the  class,  "Is  the  sun  important  for  the 
plant?"  and  "Is  the  rain  important  for  the 
plant?"  Everyone  agreed  that  both  the  sun 
and  the  rain  were  important  for  the  plant. 
Then  I replaced  the  picture  of  the  plant 
with  a human  stick  figure  and  asked  them 
to  tell  me  what  it  was.  One  student  said 
"people."  I introduced  the  word  "human 
being"  explaining  that  it  had  the  same 
meaning  as  "a  person"  or  "somebody." 
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I then  posed  the  question,  "What  is  impor- 
tant for  human  beings?"  Quite  an  exten- 
sive list  of  things  important  for  people 
was  generated  by  the  class  which  I copied 
down  on  the  chalkboard.  Among  the  ex- 
amples offered  by  my  students  of  things 
they  believed  to  be  important  for  human 
beings  were: 

sun,  water,  food,  oxygen,  other  animals, 
family,  job,  tools  [such  as  computer,  car], 
money,  education,  understanding/communi- 
cation, culture,  good  character,  religion, 
recreation,  health  [exercise,  medicine,  doc- 
tors, good  air,  good  water,  good  food],  trans- 
portation, God. 

When  asked,  "What  is  more  important, 
job  or  family?",  everyone  agreed  that  fam- 
ily was  more  important.  I asked  a few 
more  questions  contrasting  the  relative  im- 
portance of  pairs  of  the  important  things 
from  the  list  my  students  had  generated. 
These  questions  generated  some  discus- 
sion and  my  students  were  able  to  come 
to  agreement  on  answers  to  each  question. 
Finally,  I asked  "What  is  the  most  impor- 
tant?" One  of  my  students  responded  that 
God  was  most  important  because  every- 
thing else  on  our  list  of  important  things 
came  from  God.  The  rest  of  the  class 
agreed. 

At  this  point,  after  having  contemplated 
what  is  of  ultimate  importance  to  human 
beings,  the  learners  were  ready  to  contem- 
plate what  the  importance  of  learning  En- 
glish was  for  them  personally.  I posed  a 
new  question,  "Why  is  it  important  FOR 
YOU  to  speak,  read  and  write  English?" 

I emphasized  "FOR  YOU"  as  I asked  each 
individual  student  to  give  their  own  an- 
swer to  this  question.  The  students  brain- 
stormed and  I listed  their  responses  on 
the  chalkboard.  Their  responses  were  as 
follows: 
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to  communicate,  shopping  [food,  clothes, 
shoes,  car],  work,  doctor,  church,  hospital, 
telephone,  telephone  bill,  looking  for  work,  to 
progress  in  your  job,  to  go  to  the  post  office, 
immigration,  renting  an  apartment,  to  learn 
more  about  life,  to  express  my  ideas/to  talk 
about  my  ideas,  transportation  [airport,  taxi, 
bus,  go  to  car  mechanic],  recreation  [to  visit 
the  zoo,  to  go  to  the  movies],  to  learn  more 
about  US  culture  and  history,  go  to  court, 
laivyer. 

When  the  class  seemed  to  have  ex- 
hausted its  ideas,  I asked  each  student 
individually  to  pick  one  or  two  of  the  pur- 
poses for  learning  from  the  list  which  they 
felt  were  the  MOST  important  for  them 
personally.  The  result  was  that  "work" 
and  "to  express  my  ideas"  both  received 
the  highest  number  of  votes  (with  four 
votes  each)  followed  by  "shopping"  and 
"doctor"  (with  two  votes  each)  followed  by 
"hospital,"  "telephone,"  "telephone  bill," 
"looking  for  work,"  "renting  an  apart- 
ment," "immigration,"  "to  learn  more 
about  life,"  "recreation,"  "to  learn  more 
about  US  culture  and  history"  each  of 
which  received  a single  vote. 

My  learners  all  seemed  to  enjoy  and 
participate  enthusiastically  in  this  activity. 
When  one  student  asked  me  with  a smile, 
"Why  is  this  important  for  you?"  (referring 
to  this  lesson),  I answered,  "So  I can  teach 
you  better.  So  I can  teach  you  what  you 
need."  This  statement  regarding  the  pur- 
pose of  this  activity  resulted  in  further 
smiles  from  students.  In  retrospect  I see 
that  one  way  in  which  this  activity  failed  is 
that  at  the  end  of  the  activity  each  student 
did  not  have  an  individual  list  of  specific 
goals  they  wanted  to  work  towards.  I 
found  it  challenging  to  both  facilitate  the 
activity  and  at  the  same  time  keep  note  of 
what  purposes  for  learning  English  were 


important  to  which  individual  students 
and  I was  not  able  to  recall  after  the  lesson 
all  of  the  particular  goals  of  each  indi- 
vidual learner.  I was  able  to  record,  by 
copying  down  the  student-generated  lists 
that  I had  copied  onto  the  chalkboard, 
which  purposes  were  of  greatest  impor- 
tance to  the  class  as  whole.  This  informa- 
tion in  itself  was  valuable  to  guide  my 
future  curriculum  development.  Having 
an  aide  or  volunteer  in  the  classroom  dur- 
ing this  activity  to  record  each  student's  in- 
dividual responses  might  have  helped  me 
end  up  also  with  a record  of  individual 
goals /needs.  Another  possibility  which  I 
had  not  considered  but  which  I will  try  in 
the  future  is  to  have  students  write  down 
their  votes  for  their  most  important  pur- 
poses for  learning  English  and  collect  these 
at  the  end  of  the  activity. 

CONCLUSION 

To  best  serve  adult  ESOL  learners,  an 
ESOL  teacher  needs  to  acquire  the  kind  of 
information  about  each  of  his/her  learners 
that  ongoing  authentic  and  learner-cen- 
tered assessment  can  best  provide.  Upon 
acquiring  this  information,  it  furthermore 
needs  to  be  shared  with  and  understood 
by  learners  and  used  to  guide  subsequent 
curriculum  development.  Such  assessment 
should  not  be  viewed  as  separate  from 
learning  but  rather  should,  on  a regular 
basis,  be  integrated  into  learning  activities. 
It  should  identify  and  measure  learners' 
progress  towards  their  self-defined,  real 
life  needs  and  purposes  for  learning 
English.  Authentic  and  learner-centered 
assessment  involves  such  obvious  practices 
as  noticing  and  appropriately  responding 
to  the  apparent  confusion,  the  silence  or 
the  questions  of  learners  and  further  in- 
volves teachers  taking  note  of  learners' 

— — M2 


To  best  serve  adult 
ESOL  learners, 
an  ESOL  teacher 
needs  to  acquire  the 
kind  of  information 
about  each  of  his/ 
her  learners  that 
ongoing  authenitic 
and  learner- 
centered  assess- 
ment can  best 
provide.  Assess- 
ment should  not  be 
viewed  as  separate 
from  learning  but 
rather  should,  on 
a regular  basis,  be 
integrated  into 
learning  activities. 


>lnme  11:  Winter  1998 


ADVENTURES  IN  ASSESSMENT 


Volume 
11  • 66 


responses  to  learning  activities,  noticing 
how  each  learner  learns  best,  paying  atten- 
tion to  learners'  use  of  the  English  lan- 
guage as  well  as  to  the  questions  they  ask 
and  concerns  they  express.  Perhaps  most 
importantly,  such  assessment  also  neces- 
sarily involves,  and  indeed  is  based  on, 
dialogue  between  teachers  and  learners. 
Such  dialogue  should  eventually  cause 
learners  to  become  accustomed  to  monitor- 
ing and  managing  their  own  learning 
rather  than  relying  entirely  on  their  teacher 
to  do  this  for  them. 

The  LEIS  forms  and  the  BEST  test  cur- 
rently in  use  at  Wake  Technical  Commu- 
nity College  do  not  meet  the  criteria 
mentioned  above.  More  authentic  and 
learner-centered  assessment  provided  on 
an  ongoing  basis  would  clearly  be  prefer- 
able, but  before  a comprehensive,  ongoing, 
authentic  and  learner-centered  strategy  for 
assessment  can  be  implemented  in  the 
North  Carolina  Community  College  set- 
ting, the  status  of  ESOL  teachers  within  the 
North  Carolina  Community  College  Sys- 
tem in  my  opinion  would  have  to  change. 
Currently,  the  task  of  implementing  au- 
thentic and  learner-centered  assessment  in 
a comprehensive  manner  would,  I believe, 
prove  unmanageable  for  most  of  the  ESOL 
teachers  in  North  Carolina's  Community 
College  System  who  are,  as  far  as  I am 
aware,  all  part-time  teachers  who  do  not 
receive  pay  for  their  class  preparation  time 
much  less  for  the  amount  of  extra  time  it 
would  take  them  to  implement  a more 
ideal  system  of  assessment.  On  the  other 
hand,  if  teachers  were  paid  for  class  prepa- 
ration time,  then  they  could  be  expected  to 
implement  their  own  comprehensive  as- 
sessment plan  in  their  classrooms  using 
authentic  and  learner-centered  approaches 
(including  perhaps  some  of  the  assessment 
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tools  described  in  this  paper)  and  could 
justifiably  be  held  accountable  for  doing 
so.  Barring  such  a change  in  the  North 
Carolina  Community  College  System's 
policies,  the  ongoing  use  of  authentic  and 
learner-centered  approaches  to  assessment 
in  North  Carolina  community  college 
ESOL  classrooms  I am  sure  will  remain  the 
exception  rather  than  the  rule. 

North  Carolina  Community  Colleges' 
open-entry/ open-exit  policy  presents  a 
further  obstacle  to  implementing  the  kind 
of  meaningful  assessment  of  adult  ESOL 
learners  this  paper  proposes,  and,  as  I have 
explained  earlier,  this  is  most  especially 
true  in  beginning  level  ESOL  classes.  Be- 
cause this  policy  allows  new  students  to 
enter  classes  at  any  time  during  a term, 
there  is  almost  never  a time  when  every 
student  attending  a given  beginning  level 
ESOL  class  has  acquired  sufficient  English 
to  understand  and  participate  in  regular, 
learner-centered  assessment. 

Despite  these  systemic  obstacles,  I be- 
lieve ESOL  teachers  in  community  college 
settings  can  and  should  try  to  find  creative 
ways  to  provide  authentic  and  learner-cen- 
tered assessment  for  their  learners.  While 
this  probably  can  not  be  done  as  consis- 
tently and  comprehensively  as  would  be 
ideal  given  the  institutional  constraints 
these  teachers  work  under,  the  more  it  can 
be  provided  to  our  ESOL  learners,  even  to 
a small  degree,  the  better  these  learners 
will  be  served.  Informal  forms  of  assess- 
ment such  as  teacher  observations  of  learn- 
ers' authentic  use  of  English  in  the 
classroom  and  of  student  verbal  and  non- 
verbal responses  to  learning  activities  are 
perhaps  the  most  viable  means  of  authen- 
tic assessment  that  beginning  level  ESOL 
community  college  teachers  can  implement 
in  their  classrooms.  Some  initial  learning 
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activity  for  assessing  learners'  specific  pur- 
poses for  learning  English  (i.e.  communica- 
tion targets)  that  involves  dialogue  with 
and  reflection  on  the  part  of  learners  is  also 
very  important  but  again,  given  the  incon- 
sistent attendance  of  learners  and  the  lim- 
ited English  language  proficiency  of 
beginning  level  ESOL  learners  in  our  com- 
munity college  ESOL  classrooms,  it  would 
be  difficult  for  beginning  level  adult  ESOL 
teachers  to  provide  this  kind  of  assessment 
consistently  for  all  the  students  who 
choose  to  enter  their  classrooms  at  a North 
Carolina  community  college  throughout 
the  course  of  a given  term. 

In  conclusion,  authentic  and  learner- 
centered  assessment  is  vitally  important  to 
learners,  teachers  and  stakeholders  in  the 
process  of  ESOL  education.  Such  assess- 
ment is  important  to  learners  in  that  it 
helps  them  clarify  their  needs  and  goals 
for  learning  English,  gives  them  a sense  of 
their  progress  towards  these  goals  which 
they  are  intrinsically  motivated  to  attain, 
and  helps  them  evaluate  and  take  respon- 
sibility for  their  own  learning.  It  is  impor- 
tant to  teachers  in  designing,  and 
continually  re-evaluating  the  effectiveness 
of  learner-centered  curriculum  that  genu- 
inely meets  learner  needs  and  helps  them 
progress  toward  their  goals.  It  is  important 
for  administrators,  funders  and  policy- 
makers in  that  it  can  help  them  gain  a 
more  realistic  picture  of  how  the  real  out- 
comes and  effectiveness  of  ESOL  programs 
can  best  be  measured.  My  hope  is  that 


ESOL  learners,  practitioners,  and  stake- 
holders in  North  Carolina  may  work  to- 
gether towards  better  serving  the  ESOL 
learners  in  our  state  by  engaging  in  an  on- 
going evaluation  of  our  assessment  prac- 
tices and  making  the  institutional  changes 
needed  to  encourage  the  implementation 
of  truly  useful,  authentic  and  learner-cen- 
tered assessment  in  our  adult  ESOL  class- 
rooms. 
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MY  LESSON  PLAN:  WHAT  IS  IMPORTANT? 

1 . Introduce  new  vocabulary  "important/7 

Write  on  chalkboard  or  overhead,  "WHAT  IS  IMPORTANT?" 

Check  to  see  if  everyone  understands  "important."  If  not,  take  drivers  license  from  your 
pocket,  make  sure  everyone  sees  what  it  is,  and  say,  'This  is  important!  1 NEED  my  driv- 
ers license.  If  1 have  no  drivers  license,  1 can  not  drive  my  car."  Then,  take  a tissue  from 
your  pocket  and  say,  "This  is  not  important."  Saying  this,  throw  the  tissue  into  the  trash. 

Draw  a picture  of  a plant  on  the  chalkboard  and  ask,  "What  is  this?"  Then  draw  sun  and 
rain  and  ask  students  what  each  of  these  are. 

Then  ask, 

Q:ls  the  sun  important  for  the  plant?  Why? 

Is  the  rain  important  for  the  plant?  Why? 

WRITE  THESE  QUESTIONS  AND  LEARNER  RESPONSES  ON  THE  CHALKBOARD  OR 
OVERHEAD. 

2.  Replace  the  plant  in  the  picture  with  a human  stick  figure  and  ask,  "What  is  this?" 
Write  "human  being"  underneath  the  stick  figure  and  explain  the  term  "human  being." 
Then  ask  the  whole  class,  "WHAT  IS  IMPORTANT  FOR  A HUMAN  BEING?"  as  you  write 
this  question  on  the  chalkboard  or  overhead. 

Teacher:  What  is  important  for  you? 

Is  (a  job,  money,  learning  English,  watching  TV,  sleeping,  eating,  your  pencil,  your 
car,  buying  new  clothes,  your  family.  President  Clinton,  my  cat,  government,  religion, 
God  etc.)  important? 

Student:  Yes,  it  is.  (or)  No,  it  isn't,  (or)  Sometimes  it's  important. 

Teacher.  Why? 

Promote  discussion.  Create  a list  of  student  responses  to  the  question  "What  is 
important?" 

To  promote  further  discussion.  Interject  the  following  questions  as  learners  seem  to  be  run- 
ning out  of  ideas: 

Teacher:  What  is  more  important,  or  ? Why? 

Teacher:  What  is  most  important  for  you? 
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3.  Upon  completing  the  preceding  activity,  write  the  following  question  on  the  chalkboard 
and  then  read  it: 

Why  is  it  important  FOR  YOU  to  understand,  and  speak,  and  read,  and  write  English? 

Make  sure  students  understand  the  words  "understand/  "speak/  "read,"  "write."  Pro* 
mote  discussion  as  before.  Copy  student  responses  onto  chalkboard  or  overhead. 

4.  PICTURE  EXERCISE  (for  lower  level  students) 

T:  Where  is  speaking,  and  reading,  and  writing  English  important  FOR  YOU? 

(pointing  to  pictures  of  Doctor,  Telephone,  jobs  4-  Work,  Food  Shopping,  Feelings/ 

How  are  you?,  etc.) 

5.  When  the  list  of  reasons  and/or  places  where  using  English  is  important  for  your  stu* 
dents  is  complete,  ask, 

T:  Which  one  is  the  most  important? 

Ask  each  student  individually  to  vote  for  one  or  two  reasons  for  learning  English  that 
are  the  most  important  for  him  or  her.  Keep  track  of  student  votes. 
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In  the  ESOL  Classroom 


Reflections  on  Meeting  the  Challenge 
of  Assessment  with  Beginning  Students 
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Curriculum  Frameworks  Inquiry  Project,  Literacy  South,  1998,  (919)  682-8108 


by 

Cheiyl  Gant 


Wake  Technical 
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Raleigh,  NC 


1 currently  teach  two  classes.  In  the 
morning,  I teach  a Beginning  class. 
There  are  two  men  from  Congo,  one 
from  Angola,  three  from  Mexico,  three 
from  Vietnam,  one  of  whom  is  an  ethnic 
Montagnard,  and  one  from  Taiwan.  There 
are  also  three  women:  one  Palestinian,  one 
from  Somalia,  and  most  recently,  one  from 
Mexico.  The  students  range  in  age  from  21 
to  75. 

The  Palestinian  woman  and  one  of  the 
Mexican  men  never  had  formal  classroom 
training.  One  of  the  men  from  Congo  was 
institutionalized  for  a substantial  portion 
of  his  life  and  also  had  no  classroom  expe- 
rience. The  man  from  Angola  was  formerly 
a mathematics  instructor  in  his  country. 
The  Palestinian,  Mexican,  Congolese,  and 
Taiwanese  had  no  prior  formal  experience 
with  writing  using  the  English  alphabet. 

In  the  afternoon,  I teach  an  Intermediate 
class,  which  also  has  an  educationally  var- 
ied population.  At  various  points  during 
the  semester,  I have  had  two  Bulgarian 
men,  two  Indian  men,  one  Indian  woman, 
a man  from  Congo,  one  from  Togo,  a Ko- 
rean woman,  two  Brazilian  women,  two 
Mexican  men,  three  men  from  the  Domini- 
can Republic,  and  most  recently,  a woman 
from  France.  They  range  in  age  from  19  to  50. 

PROJECT  DESCRIPTION 

In  the  course  of  teaching  ESOL  formally 
over  the  past  four  years,  I have  realized 
that  our  program  lacks  repetitive  standard- 
ized testing  that  provides  teachers  with 
hard  evidence  to  support  progress  recom- 


mendations for  advancement.  Having 
come  from  a background  in  the  elementary 
schools  of  set-in-stone  curricula,  and 
"teaching  to  pass  that  math  or  reading 
exam  coming  up  in  spring"  or  "teaching 
to  the  test,"  the  idea  of  "nothing  in  stone" 
was  new  to  me.  It  did  not  occur  to  me  that 
any  other  kind  of  assessment  could  be 
valid.  With  this  in  mind,  I approached 
assessment  with  great  hesitation.  Several 
questions  came  to  mind.  How  could  I find 
where  students  really  were  in  terms  of 
their  ability?  How  could  I find. what  pro- 
gress they  had  made?  How  and  where 
would  I be  able  to  document  that  progress? 
Should  they  be  privy  to  the  information? 

I was  entering  into  a non-traditional  back- 
ground from  a traditional  one.  At  the  com- 
munity college  and  in  the  program  where  I 
work,  the  focus  is  on  helping  students  with 
life  skills.  In  the  simplest  sense,  that  meant 
not  only  was  it  our  job  to  teach  students 
the  mechanics  of  the  language,  but  more 
importantly,  expose  them  in  as  broad  a 
sense  as  possible  to  real-life  situations  and 
the  language  involved  in  functioning  in 
those  situations. 

Our  college  has  three  assessment  tools. 
The  BEST  is  used  upon  entry  into  the  pro- 
gram to  help  those  interviewing  the  new 
students  approximate  in  what  level  class 
those  students  belong.  The  next  is  an  inter- 
view form  the  students  are  required  to 
answer  verbally.  The  person  who  is  regis- 
tering the  student  asks  his/her  name, 
address,  other  personal  information,  and 
why  the  student  has  come  to  the  United 
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States.  During  the  course  of  this  interview, 
the  individual  conducting  the  interview 
decides  in  which  level  the  student  might 
be  successful.  The  last  is  a series  of  areas 
on  the  back  of  a student  information  card 
where  the  teachers  are  required  at  the  be- 
ginning and  the  end  of  the  session  to  make 
blanket  statements  about  students'  goals 
and  progress.  Often,  decisions  regarding 
the  readiness  of  our  students  to  go  on  to 
the  next  level  are  based  on  these  notes. 

The  BEST  works  very  well  for  those  stu- 
dents who  come  to  our  program  with  a 
minimal  level  of  proficiency  in  English,  but 
often  pre-beginning,  and  beginning  stu- 
dents are  not  able  to  take  the  test,  even 
though  it  does  not  reflect  their  capacity  to 
learn,  or  other  areas  of  acuity.  The  student 
information  cards  have  very  general  cat- 
egories and  no  guidelines  for  getting  infor- 
mation regarding  students'  specific  goals. 

Initially,  my  project  began  with  the 
question,  what  did  my  students  already 
know?  The  BEST  test  gives  a teacher  lim- 
ited information  at  best.  How  could  I 
record  the  amount  of  English  my  students 
already  possessed?  How  could  I help  them 
articulate  what  they  wanted  from  my 
class?  It  seemed  an  overwhelming  task, 
but  I was  willing  to  try. 

Because  my  Intermediate  students  were 
more  capable  of  expressing  their  needs,  I 
started  by  asking  them  open-ended  ques- 
tions about  what  they  thought  they  needed 
or  what  they  wanted  from  me  and  the 
class.  I attempted  to  give  them  opportuni- 
ties to  describe  anything  in  their  daily  in- 
teractions in  English  that  caused  specific 
difficulties  for  them.  I suggested  that  they 
make  a mental  or  written  note  of  these  is- 
sues and  bring  them  to  class  to  share  so 
that  everyone  could  benefit  from  them. 

Yet  from  the  beginning  of  the  project, 
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I had  a sinking  feeling  that  there  was 
something  I was  doing  of  which  I wasn't  en- 
tirely aware.  This  led  me  to  do  some  reading 
about  what  could  be  considered  assessment. 
In  the  midst  of  this  process  my  focus  changed 
from  trying  to  find  out  students'  goals  to 
determining  which  kind  of  assessment 
would  allow  me  deeper  insight  into  how 
the  process  can  be  effectively  carried  out. 

In  the  project,  I decided  instead  to  deter- 
mine which  tools  are  the  most  feasible  for 
assessing  and  making  the  case  for  what 
worked  best  in  my  personal  teaching  situa- 
tion. 

The  importance  of  this  project  has  two 
parts.  First  of  all,  I hope  that  by  looking  at 
what  can  be  done  to  get  at  students'  goals, 
we  will  eventually  change  the  system  we 
currently  use  to  record  those  goals  and 
have  them  articulated  by  students  them- 
selves. Second,  it  is  essential  that  we  be 
able  to  better  address  the  specific  problems 
that  are  inherent  in  charting  the  progress 
of  and  meeting  the  specific  needs  of  our 
entry-level  students  with  practical  assess- 
ment tools. 

PUTTING  OUT  FEELERS 

Since  I came  with  a limited  view  of  any 
other  kind  of  assessment  and  no  experi- 
ence in  how  to  carry  it  out,  I took  to  read- 
ing about  different  types.  I discovered  with 
great  interest  that  there  are  several  kinds  of 
assessments  included  among  those  classi- 
fied as  "alternative."  Alternative  assess- 
ment is  based  on  daily  classroom  activities 
and  "includes  a variety  of  instruments  that 
can  be  adapted  to  varying  situations" 
(Huerta-Macias,  1995).  This  made  sense  to 
me.  So  again,  the  question  of  how  to  do 
this  with  beginners  came  up.  I am  trying  to 
look  at  where  they  are  when  they  put  their 
feet  in  the  door.  What  indicators  should  I 
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I had  been  working 
under  the  assump- 
tion that  any  kind 
of  assessment 
would  have  to  be 
formal,  in  some 
kind  of  question- 
naire form,  and 
that  I would  have 
to  have  masses 
of  those  question- 
naires repeated 
over  time  for  them 
to  be  valid. 


be  looking  for? 

I also  set  out  to  gather  information  on 
indicators  into  the  goals  of  emergent 
speakers  and  writers.  Was  there  a survey 
that  one  could  administer  to  pre-beginning 
or  pre-literate  students?  Many  of  the  tasks 
I read  about  were  for  students  who  could 
already  complete  coherent  sentences. 
Among  the  methods  that  I read  about 
were:  student  portfolios,  learner  logs,  cloze 
activities,  learner  grids,  learner  profiles, 
teacher  observations,  and  formal  surveys. 
While  reading  about  these  tools,  I realized 
that  many  teachers  in  our  program  had 
used  the  most  limited  and  probably  the 
least  productive  of  these  methods,  or  we 
had  not  validated  the  methods  that  were 
being  used.  I discovered  that  the  student's 
same  day-to-day  activities  (e.g.  writing, 
role-playing,  group  discussion)  are  the 
basis  for  alternative  assessment  (Huerta- 
Macias,  1995). 

I had  been  working  under  the  assump- 
tion that  any  kind  of  assessment  would 
have  to  be  formal,  in  some  kind  of  ques- 
tionnaire form,  and  that  I would  have  to 
have  masses  of  those  questionnaires  re- 
peated over  time  for  them  to  be  valid.  The 
more  I read,  however,  the  more  I discov- 
ered that  many  other  teachers  and  I had 
been  charting  students'  progress  through 
personal  observation.  In  one  piece  I read 
that  discussed  using  reader  logs  as  assess- 
ment tools,  the  writer  admitted  that  the 
instructor  found  lower-level  learners  had 
difficulty  responding.  Very  often  the  state- 
ment "some  things  I like"  was  answered 
"I  like  everything"  (Gear,  1993).  It  is  this 
kind  of  generality  in  students'  surveys 
with  open-ended  questions  that  makes 
assessing  beginning  students  particularly 
frustrating.  This  issue  is  complicated  if  the 
students  cannot  understand  the  scope  of 
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what  is  being  asked  of  them,  and  there  are 
no  translation  helps  in  the  class  because  no 
one  else  speaks  the  person's  language. 

The  same  article  mentions  another  really 
helpful  assessment  tool:  an  individual  exit 
interview  with  the  instructor  (Gear,  1993). 
In  our  program,  students  complete  an 
evaluation  of  the  teacher  at  the  end  of 
each  15-17  week  session.  The  evaluation 
includes  questions  about  the  teacher's 
preparedness,  how  well  she  has  helped 
the  students  reach  their  goals,  and  her 
demeanor.  The  language  on  the  survey  is 
problematic  for  beginning  students  if  they 
don't  have  an  interpreter.  It  has  an  area  for 
comments  but  the  survey  is  more  for  infor- 
mation than  a measure  of  the  students' 
understanding  of  what  they  learned. 

Complicating  the  issue  of  assessment  on 
all  levels  is  our  attendance  policy,  which  is 
open  entry,  allowing  for  much  elasticity  in 
a student's  attendance  - perhaps  too  much 
to  get  an  accurate  measure  of  where  our 
students  started  and  where  they  end  up  at 
the  end  of  a session. 

Encased  in  any  assessment  activity 
should  be  activities  designed  to  help  stu- 
dents become  self-aware.  One  of  the  mis- 
sion statements  of  assessment  should  be 
that  self-assessment  is  highly  important  for 
successful  language  learning  (McNamara, 
Dean,  1995).  By  working  with  what  a stu- 
dent thinks  he  wants  to  learn,  we  can  help 
students  establish  realistic  expectations 
about  what  language  skills  they  need  to 
achieve  their  goals  (McNamara,  Dean, 
1995).  The  most  important  piece  is  helping 
students  establish  their  expectations.  As- 
sessment is  needed  to  help  them  develop 
internal  indicators  for  where  they  stand  in 
their  ability.  It  also  makes  it  imperative  to 
include  them  in  the  assessment,  not  as 
guinea  pigs,  but  as  active  participants  in 
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its  shaping.  The  challenge  in  this  at  the  be- 
ginning level  is  that  it  may  take  more  time 
for  them  to  leam  the  language  needed  to 
articulate  if  they  like  the  style  of  question- 
ing than  there  is  time  to  complete  the  ques- 
tions themselves.  A way  around  this  is 
performance-based  assessments  that  ask 
learners  to  complete  tasks  or  take  part  in 
simulations  (Spruck,  Wrigley,  1992).  Any 
kind  of  role  playing  or  oral  testing  as  well 
as  writing  of  basic  information  would  be 
assessment  activities  with  documentable 
results. 


TRIAL  ...  AND  ERROR  ...  AND  TRIAL 

With  this  information  in  hand,  I decided 
to  look  at  the  two  types  of  assessment  I 
used  most  in  my  practice:  informal  stu- 
dent-teacher discussions  on  what  students 
felt  was  important,  and  open-ended  ques- 
tions. Since  I began  my  project  with  my  in- 
termediate students,  I would  like  to  first 
share  what  questions  were  discussed  with 
them. 

In  the  first  three  weeks  of  my  intermedi- 
ate class,  I posed  several  questions  to  the 
students  to  have  them  consider  what  they 
wanted  to  be  able  to  do  with  the  language. 
In  the  past,  I had  literally  asked  students 
what  they  wanted  to  leam  and  they  would 
give  me  some  very  general  answer  like 
"grammar"  or  "pronunciation."  I believe  a 
language  can't  be  learned  just  by  knowing 
the  mechanics.  One  must  understand  the 
context  in  which  language  is  used.  In  my 
intermediate  classes  I constantly  attempt  to 
get  students  to  understand  that  they  know 
many  of  the  mechanics  to  speak  already. 
What  is  needed  are  the  contexts  in  which 
to  use  the  language,  and  fine-tuning  of 
skills. 

On  the  board,  I wrote  the  following 
questions: 
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1)  I wish  I could in  English. 

2)  If  I knew  how  to in  English  it 

would  be  great. 

3)  What  I want  to  know  how  to  say 

is . 

I expected  answers  that  were  very  broad 
and  non-specific.  In  this  particular  exer- 
cise, I didn't  ask  students  to  write  anything 
down,  but  to  discuss  their  answers.  Here 
are  some  of  the  things  they  said: 

1)  I wish  I could in  English. 

sing-  know  songs 
solve  ex-wife  problems 
tell  romantic  stories 

talk  about  sports 

talk  about  what  I do  in  my  country 

understand  news 
understand  TV  show 
describe  what  I'm  thinking 

tell  jokes  say  the  names  of  spices 

make  a woman  fall  in  love  with  me 

2)  If  I knew  how  to in  English 

it  would  be  great. 

get  involved  in  English  conversations 
with  Americans 

talk  about  philosophy 

3)  What  I want  to  know  how  to  say 

is . 

the  way  I feel  in  class 
what  to  say  for  a job  interview 

friendly  words 
words  to  make  happy 

I was  really  surprised  at  the  things  that 
students  expressed  an  interest  in.  Not  one 
student  mentioned  grammar.  They  all 
talked  about  how  they  could  fit  into  the 
scheme  of  things  in  their  lives  here  in  the 
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U.S.  Perhaps  answers  of  this  nature  are 
based  on  difficult  experiences  students  had 
when  first  trying  to  adjust  to  their  new 
lives.  It  may  be  that  the  more  opportunities 
for  social  interaction  that  students  have, 
the  more  they  see  the  need  for  practical 
uses  of  English. 

Every  week  I also  asked  them  questions 
about  what  they  had  learned,  what  they 
liked  and  didn't  like.  I didn't  find  this  line 
of  questioning  to  be  as  successful. 

I would  post  the  following  questions  on 
the  board  at  the  end  of  the  week: 

1)  This  week  I learned  about . 

2)  I liked . It  helped  me  a lot. 

3)  I didn't  like . It  didn't  help 

me. 

Eliciting  answers  from  these  questions 
was  considerably  more  difficult.  I'm  not 
sure  if  it  was  because  I was  immediately 
throwing  students  into  a self-assessment 
role  or  if  their  expectations  were  still  un- 
clear to  me. 

1)  This  week  I learned  about . 

who  is  everybody  in  class 
body  language/gestures 
voice  register  pronouns 

2)  I liked . It  helped  me  a lot 

learning  the  names 

asking  questions  about  American  culture 

everything 

3)  I didn't  like . It  didn't  help. 

too  much  talking 

At  the  end  of  the  third  week,  attendance 
went  from  18  students  in  my  intermediate 
class  to  three.  I thought  perhaps  I had 
overwhelmed  them  with  the  line  of  ques- 
tioning I had  taken.  Normally,  part  of  the 
introductory  process  in  my  class  includes 
ice-breaking  activities.  I insist  that  students 
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get  to  know  one  another  so  they  are  not 
strangers.  It  takes  about  that  three-week 
period  for  them  to  feel  comfortable  with 
each  other  and  with  the  notion  of  becom- 
ing self-aware.  It  also  takes  me  about  that 
amount  of  time  to  get  an  idea  about  which 
direction  to  take  my  class. 

I didn't  realize  at  the  time  that  I was 
constantly  making  mental  notes  about  ev- 
erything involving  my  students:  who  was 
very  talkative,  who  wasn't,  and  who 
seemed  willing  to  engage  in  class  activities 
which  might  seem  unusual  to  them.  I 
noted  who  thought  my  speech  was  too 
fast,  who  acknowledged  it,  and  who  felt 
constrained  that  I had  a special  role  or 
place  as  "the  teacher."  I paid  close  atten- 
tion to  who  seemed  willing  to  talk  about 
their  personal  lives  and  who  was  ex- 
tremely reticent  or  evasive.  I watched  for 
signs  of  boredom,  involvement,  and  lack  of 
comprehension  through  facial  expressions 
and  body  language.  I paid  attention  to 
who  constantly  felt  the  need  to  speak  in 
their  own  language  if  they  came  with  a 
companion  from  their  country  and  who 
seemed  reluctant  to  ask  questions. 

This  is  a process  I have  engaged  in  since 
I began  teaching  ESL,  but  because  it  never 
involved  written  tests  of  any  kind,  I didn't 
even  consider  that  I was  doing  assessment. 
To  my  misfortune,  I didn't  record  my 
notes  because  I considered  them  for  my 
personal  use  only  and  not  very  effective  in 
helping  anyone  other  than  me.  I learned, 
however,  that  this  information,  even 
though  not  officially  recorded,  was  viable 
to  use  for  assessment,  and  in  the  future, 
worth  mapping  out.  I realized  also  that, 
based  on  my  personal  observation  of  stu- 
dents, I was  making  decisions  about  what 
they  were  interested  in  studying  and  how 
capable  or  willing  they  were  to  advance  in 
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dealing  with  one  American.  In  the  face 
of  working  with  three  or  four  students,  I 
didn't  think  I had  enough  to  make  a real 
case  for  this  kind  of  assessment. 

I then  thought  to  modify  the  open- 
ended  question  format  for  my  beginning 
class,  which  had  much  more  consistent 
attendance.  I wrote  on  the  board: 

Hike . 

I don't  like . 

In  English  I can . 

I want . 

I don't  want . 

I explained  "like"  with  a gesture  of 
thumbs  up  and  synonyms  such  as  good 
and  okay,  and  I explained  "don't  like" 
with  a thumbs  down  sign  and  synonyms 
such  as  bad  and  shook  my  head  in  the 
negative.  To  help  them  understand  "can", 

I gave  them  examples  of  what  they  could 
do,  such  as  speak  in  their  native  languages, 
and  cook  or  work,  depending  on  what  the 
students  seemed  to  express  as  their  inter- 
ests. It  was  an  activity  in  itself  to  get  the 
students  to  talk  about  themselves. 

Their  answers  to  these  open-ended 
questions  were  varied.  One  of  the  Mexican 
men  wrote: 

"I  like  weldin,  money,  dance  worken 
(working). 

I don't  like  cold,  rainy. 

In  English  I can  yes,  okay,  I like. 

In  English  I cannot  pronunciation. 

The  Somalian  woman  wrote: 

I like  my  eyes. 

I like  mosque. 

I wan  to  go  school 

I don't  want  club 

I don't  want  cinema 

The  Montagnard  man  wrote: 

I like  com. 

I like  put  a pant. 

d 


I don't  like  eat  mushrooms 
In  English  I cannot  say  understand 
I want-say  I love  you 
I don't  want  eat  chicken 

The  Taiwanese  man  wrote: 

I like  woman-girl-mother 
I don't  like  winter-paint-shovel 
I can  cut  cook  clear  trim" 

I told  them  they  could  use  picture  dic- 
tionaries to  carry  out  this  exercise.  During 
its  course,  I observed  them.  Who  took  the 
initiative  to  tell  me  they  were  stuck  or  un- 
clear about  the  exercise,  and  who  just 
waited  for  the  end  of  the  exercise  to  see  the 
results  of  everyone  else?  Who  wanted  me 
to  look  immediately  at  their  work,  and 
who  had  reservations?  It  seemed  these 
types  of  activities,  while  the  activity  itself 
was  important,  were  just  additional  ways 
to  determine  who  seemed  motivated  and 
unafraid,  and  who  needed  motivation.  I 
was  frustrated  with  the  generalities  of  the 
answers  and  looked  for  another  way  to  get 
at  exactly  what  the  students  thought  they 
were  supposed  to  be  getting  from  English 
class. 

Trying  to  help  students  feel  comfortable 
enough  to  express  their  needs,  I presented 
myself  not  as  the  authority  of  the  class- 
room, but  only  as  the  authority  in  the  lan- 
guage. I led  them  in  modeled  activities, 
which  included  talking  about  themselves 
and  their  everyday  experiences,  by  sharing 
my  life  with  them.  I shared  what  I thought 
was  good  and  bad  and  encouraged  them 
to  do  the  same.  I was  not  sure  that  the 
example  I was  trying  to  set  was  having 
an  impact  until  something  amazing  hap- 
pened. One  of  the  Mexican  men  who  had 
recently  joined  the  class  said  one  day  that 
he  wanted  to  learn  how  to  get  food  at 
McDonald's.  He  didn't  exactly  use  that 
terminology.  It  was  more  like  "Eat 
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While  recording 
these  reflections, 
I have  come  to 
realize  that  the 
most  comprehen- 
sive form  of 
assessment  that 
I have  as  an 
instructor  is 
my  personal 
observations. 


McDonald's  how?"  From  his  request  came 
a two-week  lesson  on  ordering  food  from 
fast  food  restaurants,  which  included  vo- 
cabulary, speaking  and  listening,  and  writ- 
ing practice.  Even  the  most  disengaged 
students  became  more  active  in  their  par- 
ticipation. After  this  incident  my  beginning 
students  started  to  ask  simple  questions 
about  how  to  spell  certain  words.  Even 
the  Taiwanese  man  lost  his  hesitation  and 
would  stop  the  lessons  in  the  middle  of  a 
word  or  sentence  to  clarify  a letter  and  the 
sound  that  went  with  it.  One  of  the  other 
Mexican  men  took  the  initiative  to  be  the 
server  and  waited  on  the  whole  class. 

I repeatedly  teach  words  used  to  ask 
questions  in  my  beginning  class  in  an  ef- 
fort to  get  them  to  understand  what  people 
are  asking  them.  It  is  also  to  help  them  un- 
derstand how  to  ask  questions  themselves. 
Part  of  my  observation  at  the  close  of  the 
session  has  to  do  with  who  seems  to  have 
overcome  fear  of  mistakes  and  who  has 
learned  to  encourage  his  fellow  classmates. 
It  may  not  seem  that  this  is  related  to  as- 
sessment. Yet  as  part  of  teacher  observa- 
tion, forward  movement  of  this  kind  is 
crucial  in  determining  how  far  students 
have  come  in  their  overall  communication 
skills.  It  suggests  a comfortability  with  the 
language  that  cannot  be  quantified  on  any 
kind  of  standardized  testing. 


CLOSING  STATEMENTS: 

A WORK  IN  PROGRESS 

While  recording  these  reflections,  I have 
come  to  realize  that  the  most  comprehen- 
sive form  of  assessment  that  I have  as  an 
instructor  is  my  personal  observations.  It 
is,  however,  lacking  in  some  components. 
In  one  of  the  articles  I perused  in  my  re- 
search, one  practitioner  observed  that  she 
had  not  asked  the  learners  to  help  develop 
this  list  of  strategies  with  her.  Therefore 
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she  risked  asking  them  to  measure  them- 
selves against  objectives  which  may  not  be 
realistic  goals  for  them  or  which  simply 
may  not  be  their  goals  (Barry,  1993).  My 
observations  as  a teacher  are  not  so  much 
an  accurate  reflection  of  students'  goals, 
but  rather  of  the  standards  by  which  I de- 
termine where  students  are  in  their  ability 
to  handle  the  language.  It  would  be  effec- 
tive in  the  future  to  record  these  questions 
on  a graph  or  chart  and  expose  them  to  the 
class  as  the  pointer  that  I use  to  figure  out 
if  they  are  ready  to  go  on  to  the  next  class. 

In  this  way,  if  they  disagreed  with  my  line 
of  reasoning,  they  could  become  active 
participants  in  revising  the  questions.  This 
would  work  particularly  well  to  integrate 
students'  self  articulated  goals.  In  the  case 
of  upper  level  learners,  this  approach 
would  make  the  most  sense.  With  begin- 
ning learners,  however,  it  seems  that  it 
would  be  more  effective  for  the  teacher  to 
make  a chart  with  pictures  that  depicted 
students'  progress  in  functional  areas,  such 
as  writing  the  entire  alphabet,  or  memoriz- 
ing their  address  and  phone  number  cor- 
rectly, or  learning  to  ask  for  food  or 
directions  in  a grocery  store. 

No  matter  what  form  teacher  observa- 
tions take,  teachers  in  any  program  should 
collectively  note  what  they  observe  and 
pool  the  information  to  create  a document 
or  documents  that  could  be  circulated 
throughout  the  levels,  along  with  the  cur- 
rently used  form  as  a guideline  for  docu- 
menting ongoing  assessment.  This  last  step 
would  serve  to  deflect  any  criticism  that 
observation  is  not  a documentable  means 
of  assessment.  Since  viable  documentation 
should  be  part  of  the  goal  of  assessment,  it 
makes  sense  to  have  a form  that  consists  of 
teacher  and  student  observations.  The  cur- 
rent information  in  use  is  much  too  broad 
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to  reflect  what  specific  progress  has  been 
made. 

Something  else  to  consider  is  that  obser- 
vations should  be  continuous,  not  neces- 
sarily at  measurable  intervals.  By 
recording  segments  of  conversations  with 
students  over  time,  a teacher  can  gather  a 
true  reflection  of  their  views  of  themselves 
as  learners  (Barry,  1993).  It  isn't  necessary 
that  a log  of  these  conversations  be  for- 
mally-recorded events,  but  that  they  are 
annotated  according  to  the  date  that  they 
take  place.  It  could  be  more  a jotting  down 
of  impromptu  blurbs. 

I come  away  from  this  project  with  a 
clearer  knowledge  of  how  to  enable  my 
students  to  be  aware  of  their  desires  re- 
garding English,  other  than  just  knowing 
the  structures  of  the  language.  I also 
gained  insight  into  how  to  record  my  stu- 
dents' progress  using  tools  which  I already 
possess  and  use  on  an  informal  but  ongo- 
ing basis.  I have  acquired  a vision  of  how 
our  current  assessment  tools  can  be 
adapted  and/or  modified  to  help  teachers 
better  capture  their  students'  progress.  It 
would  also  be  helpful  for  teachers  to  take  a 
bigger  stake  in  their  students'  progress  by 
conducting  interviews  with  them  on  an  in- 
dividual basis  at  the  end  of  the  session  to 


see  where  each  student  stands,  and  to 
record  the  outcome  of  this  interview. 

My  hope  is  that,  in  spite  of  any  obstacles 
that  are  inherently  a part  of  programs  such 
as  this,  other  ESOL  instructors  will  see  that 
it  is  possible  to  determine  the  needs  of  the 
moment  from  their  students,  no  matter 
what  their  level. 
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In  conducting  action  research,  I in- 
tended to  demonstrate  the  limitations 
of  the  standardized  testing  (English  as 
a Second  Language  Oral  Assessment)  used 
in  our  program.  I approached  the  research 
with  a preconceived  opinion  that  a variety 
of  more  suitable  tests  would  provide  valid 
information  about  our  students  and  be 
learner-centered.  Based  on  my  own  dissat- 
isfaction with  this  test  — a feeling  shared 
by  the  majority  of  teachers  — I believed  I 
could  develop  a more  appropriate  test 
which  would  meet  the  needs  of  all. 

As  I continued  my  research  a whole  new 
perspective  emerged.  The  fault  in  the  pro- 
cess was  not  the  test  itself;  in  fact  the  test 
was  suited  for  its  intended  purpose.  The 
use  of  the  ESOLA  was  primarily  a book- 
keeping measure,  another  piece  of  account- 
ability which  staff  members  "expected"  to 
provide  information  about  the  learner  in 
the  program 

I reviewed  my  program  and  saw  how  it 
can  be  improved.  During  our  future  pro- 
gram development  time,  the  teachers  and 
I will  review  how  to  develop  authentic  as- 
sessment which  can  be  used  in  conjunction 
with  standardized  testing  to  provide  teach- 
ers with  information  they  need  to  assess 
the  progress  of  students  and  the  success  of 
the  curriculum. 

RESEARCH  QUESTION: 

HOW  IS  THE  ESOLA  TEST  BEING  USED 
IN  THE  NEW  BEDFORD  ADULT 
EDUCATION  PROGRAM? 

In  a decade  when  educational  account- 
ability is  expected,  many  adult  education 
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programs  have  sought  to  document  learn- 
ers' progress  and  ensure  future  funding 
through  standardized  tests.  Given  this  pre- 
vailing climate,  the  New  Bedford  Adult 
Education  Program  incorporated  the  use  of 
the  ESOLA  for  initial  student  placement 
and  as  a periodic  evaluation  to  document 
student  progress.  As  stated  in  the  ratio- 
nale of  the  test  booklet  this  test  could  be 
used  to  determine: 

1.  Student  entry  level 

2.  Student  progress  level 

3.  Student  ability  to  follow  directions 

4.  Student  ability  to  follow  English  pat- 
terns 

5.  Students  ability  to  use  specific  basic 
vocabulary. 

What  emerged  from  a periodic  adminis- 
tration of  this  test,  however,  was  teacher 
dissatisfaction  with  repeated  use  of  a 
single  test  which  provided  little  or  no  in- 
formation on  learner  progress.  To  address 
the  issue  of  ESOLA's  effectiveness,  it  is 
necessary  to  examine  how  the  test  is  used 
for  student  placement  and  to  evaluate  stu- 
dent progress. 

My  inquiry  began  with  research  in  how 
the  ESOL  literacy  is  currently  assessed.  I 
looked  at  three  articles  "Learner  Assess- 
ment in  Adult  ESL  Literacy"  by  Heide 
Spruck  Wrigely,  "Adult  ESL  Learner  As- 
sessment Purposes  and  Tools"  by  Miriam 
Burtand  and  Fran  Keenan,  and  "Assessing 
Integrated  Language"  by  Deborah  Short. 

As  stated  in  the  research,  standardized 
tests  are  cost  effective,  time  efficient  and 
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meet  documentation  needs.  But  by  their 
very  nature  are  one  dimensional  activities 
which  cannot  measure  the  complexity  of 
language  or  learning.  A detailed  review  of 
the  ESOLA  illustrates  that  it  does  meet  the 
criteria  of  a standardized  test.  It  utilizes 
basic  vocabulary:  chair,  coat,  ear.  It  in- 
creases in  difficulty  as  you  progress 
through  four  levels.  It  does  address  En- 
glish language  patterns  and  the  ability  to 
follow  directions.  It  does  deal  with  basic 
and  advanced  language  skills  and  its  pro- 
gression in  literacy  is  logical. 

What  it  lacks,  however,  is  the  use  of  any 
simulated  real  life  activities,  such  as  social 
conversation,  reading  food  labels,  or  writ- 
ing simple  sentences.  Nonetheless,  it  re- 
mains, theoretically,  a valid  instrument 
based  on  sound  educational  research. 

I then  distributed  teacher  and  student 
surveys  (see  Figure  A)  to  determine 
whether  the  ESOLA  test  provides  accurate 
initial  information  for  student  placement. 
These  surveys  and  interviews  conducted 


suggest  that  the  ESOLA  has  been  a reliable 
initial  placement  instrument.  The  test 
could  be  expanded  to  include  writing 
samples  for  more  advanced  students,  but 
for  placement  purposes  it  is  sufficient. 

The  ESOLA  is  also  used  to  periodically 
evaluate  student  progress;  it  is  adminis- 
tered twice  annually.  Primarily  used  to 
meet  program  requirements,  it  is  a numeri- 
cal indicator  of  student  success  as  listed  by 
increasing  test  score  to  insure  future  fund- 
ing. 

Since  assessment  is  an  important  part  of 
learning,  teachers  view  assessment  as  a 
method  of  evaluating  the  success  of  their 
curriculum  and  as  a method  of  meeting  the 
needs  of  their  students.  At  present,  they 
view  the  administration  of  the  ESOLA  as  a 
monotonous  process  which  yields  only  a 
flat  number  score.  What  is  more,  the  basic 
vocabulary  and  language  patterns  used  in 
the  test  are  not  addressed  in  the  present 
curriculum.  This  is  an  artificial  testing  situ- 
ation. 


FIGURE  A 

TEACHER  SURVEY  #1 
NAME 

STUDENT  SURVEY#! 
NAME 

Using  a scale  of  1-6  please  answer  the  following  ques- 

1. Do  you  like  the  class  you  are  attending?  Yes  No 

tions? 

2.  Do  you  feel  that  the  teacher  can  help  you  learn? 

12  3 4 5 6 

Yes  No 

not  at  all  somewhat  neutral  at  times  yes  definitely 

3.  Do  you  feel  this  class  is  too  easy?  Yes  No 

4.  Do  you  feel  this  class  is  too  hard?  Yes  No 

1 . Do  you  feel  that  the  present  student  placement  pro- 

5. Would  you  like  another  class  better?  Yes  No 

cess  is  accurate? 

2.  Do  you  feel  that  the  ESOLA  test  is  an  adequate  indi- 

cator of  a student's  level? 

3.  Do  you  find  yourself  moving  students  to  another 

class  soon  after  placement? 
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The  entire  process 
of  reporting 
student  progress 
needs  to  be 
expanded  so  that  it 
will  not  only  show 
minimal  gains 
but  actually  reflect 
all  the  personal 
achievement  made 
during  the 
academic  year. 


In  order  to  illustrate  the  test's  shortcom- 
ings, I will  talk  about  two  learners.  One 
student  attended  faithfully  for  a two-year 
period  and  made  significant  oral  language 
gains.  Yet  her  test  scores  indicate  only 
slight  progress.  Another,  who  attended 
faithfully  for  a three-year  period,  used  her 
skills  to  obtain  citizenship  and  a job  pro- 
motion. Yet  her  scores  indicate  erratic 
progress  and  some  regression  during  this 
time. 

In  order  to  clarify  this  situation,  I con- 
ducted an  individual  survey  with  four 
teachers  and  eight  students.  The  following 
questions  were  used 

Teacher  interview 

Do  you  enjoy  administering  the  ESOLA 
test? 

Do  you  feel  it  provides  you  with  useful 
information? 

Do  you  feel  it  provides  you  with  accu- 
rate information? 

Do  you  feel  it  reflects  what  you  teach? 

Can  you  use  this  test  to  improve  teach- 
ing techniques? 

Does  it  meet  your  students'  goals? 

Student  interview 

Do  you  like  the  test? 

Do  you  feel  it  helps  you? 

Why  do  you  take  the  test? 

The  results  were  hardly  surprising. 

Both  teachers  and  students  failed  to  see  the 
correlation  between  the  testing  and  the  in- 
struction occurring  in  the  classroom. 

If  periodic  assessment  is  to  be  useful  to 
the  teacher  and  the  student,  the  ESOLA 
must  be  expanded  to  include  much  more. 

An  analysis  of  student  writing  samples, 
portfolios,  and  response  journals  can  be 
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used  to  supplement  the  assessment  pro- 
cess. Personal  goals  sheets  along  with  vo- 
cabulary lists  can  also  show  gains  in 
classroom  performance.  Teachers  might 
choose  to  develop  their  own  tests  based  on 
the  curriculum  which  will,  in  turn,  lead  to 
better  classroom  teaching  and  increased 
student  learning.  The  entire  process  of  re- 
porting student  progress  needs  to  be  ex- 
panded so  that  it  will  not  only  show 
minimal  gains  but  actually  reflect  all  the 
personal  achievement  made  during  the 
academic  year. 

Despite  its  limited  testing  features,  the 
New  Bedford  Adult  Education  Program 
remains  highly  successful.  Students  attend 
140-155  hours  a year.  Personal  goals  are 
met  as  students  gain  valuable  life  skills 
and  move  from  level  to  level.  This  year 
120  students  successfully  completed  three 
years  in  the  program. 

In  conclusion,  is  the  ESOLA  test  an  ef- 
fective instrument  in  student  placement? 
Generally,  yes.  Is  the  ESOLA  test  as  effec- 
tive instrument  in  monitoring  student 
progress?  No,  it  is  not.  It  barely  meets  pro- 
gram requirement  and  does  not  provide 
any  additional  information  on  curriculum 
or  student  progress.  There  is  a definite 
need  in  our  program  to  reexamine  the  as- 
sessment process  and  to  make  it  more 
valid.  The  ESOLA  was  not  designed  with 
The  New  Bedford  Adult  Education  Pro- 
gram in  mind.  It  is  at  this  point  that  we, 
the  program,  can  begin  to  review  the  as- 
sessment process.  This  is  a definite  pro- 
gram need.  Teachers  need  time  to  address 
this  issue  and  develop  consistent  strategies 
to  complete  the  assessment  portion.  With 
this  conclusion,  I know  which  areas  I need 
to  address  for  futue  program  develop- 
ment. 
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Review 

New  Ways  of  Classroom  Assessment 


NEW  WAYS  OF  CLASSROOM 
ASSESSMENT,  J.D.  Brown  Editor, 

Teachers  of  English  to  Speakers  of  Other 
Languages  (TESOL),  1600  Cameron  St., 
Suite  300,  Alexandria,  VA  22314 

When  I first  began  reading  the  intro- 
duction to  this  text.  I learned  its 
Y Y contents  would  be  a series  of  ac- 
tivities pertaining  to  assessing  a variety  of 
skills  and  self  assessments  by  the  learners. 

I thought  it  would  be  just  another  book 
filled  with  lesson  plans.  The  facilitators  of 
the  ABE/ESOL  ACCCESS  classes  have 
specific  questions  about  assessment.  They 
want  to  know  explicitly  what  a portfolio  is, 
how  it  can  successfully  fit  into  classroom 
learning.  I was  pleasantly  surprised  when  I 
looked  closer  that  many  of  my  assessment 
questions  were  answered  in  well-detailed 
activities  that  clearly  demonstrate  alterna- 
tive methods  of  assessment. 

I felt  that  the  best  way  to  review  a text  of 
this  kind  was  to  put  some  of  the  activities 
into  practice  in  the  classroom.  I asked  some 
of  the  ABE/ESOL  facilitators  in  the 
ACCCESS  program  to  test  the  assessments. 
In  giving  assignments  I tried  to  vary  the 
activities  so  there  would  be  representation 
from  each  of  the  five  parts  of  the  book.  The 
facilitators  critiqued  their  activities  in 
terms  of  how  they  worked,  whether  they 
eliminated  or  added  anything,  and  the  re- 
action from  their  learners.  (See  individual 
facilitator  notes  at  the  end  of  this  section.) 

The  assessment  activities  in  this  book 
could  be  a starting  point  for  ESOL  facilita- 
te 
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tors  to  develop  further  any  ideas  that  will 
enhance  the  language  process  ( reading, 
writing,  listening,  speaking)  in  their  indi- 
vidual classes.  Learners  Access  to  Assess- 
ment, p.  60,  works  well  with  ESOL  learners 
because  it  is  ongoing  and  is  built  into  class- 
room tasks.  Learners  are  willing  to  partici- 
pate and  the  reinforcement  of  including 
the  entire  learning  process  (listening, 
speaking,  reading  and  writing)  is  a good 
reminder  for  facilitators  that  this  should  be 
ongoing. 

Activities  in  this  text  are  more  challeng- 
ing and  creative  than  activities  found  in 
most  ESOL  textbooks.  Many  resources  for 
the  ESOL  learner  do  not  ask  the  learner  to 
"stretch"  their  skills  beyond  class  activities 
and  facilitators  are  constantly  looking  to 
plan  activities  that  assess  their  students 
progress/skills.  This  text  presents  ideas/ 
methods  that  accomplish  both. 

Questions  for  pre-goal  setting,  appendix 
guidelines,  and  sample  self-assessment 
questions  found  in  the  majority  of  the 
activities  are  well  defined.  Options  for 
portfolio  assessments  — one  in  particular 
— allow  learners  to  write  additional  read- 
ing exercises  which  could  develop  into  a 
learner  text  to  be  used  in  a program. 
GREAT! 

In  our  experience,  ESOL  learners  enjoy 
assessing  their  own  oral  language  process 
by  using  videos.  There  is  a good  procedure 
for  beginners  relating  to  videos  and  oral 
language  use.  Most  of  the  ESOL  learners 
in  our  program  also  like  their  facilitator 
to  share  in  the  assessment  process. 

— — &n 
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The  text  does  need  to  address  the  entry/ 
beginner  levels  more  often.  Those  listed  as 
such  in  this  text  could  not  be  performed 
comfortably  by  our  learners.  Some  assess- 
ments are  too  difficult  to  put  into  practice 
because  they  are  developed  for  more  "uni- 
versity level"  ESOL.  Some  are  also  very 
time  consuming.  I found  some  of  the  infor- 
mation in  the  appendix  is  presented  in  a 
negative  manner.  Some  cultures  are  sensi- 
tive to  how  evaluation/ assessment  is  con- 
ducted, whether  by  a peer  or  a facilitator 
(e.g.  Appendix,  p.  101  — Self  Peer/Teacher 
Assessment  of  Group  Performance,  and 
Appendix,  p.  86  — Test  Your  Talk). 

Overall,  the  volume  is  good.  The  edi- 
tor's notes  at  the  beginning  of  each  section 
are  well  written  and  informative  (espe- 
cially helpful  for  the  new  ESOL  practitio- 
ner) . He  weighed  both  sides  of  an 
assessment  issue,  the  advantages  as  well  as 
the  disadvantages  for  assessing  a particu- 
lar way. 

The  following  are  the  comments  on  spe- 
cific tools  as  written  by  the  ACCCESS  staff. 
From  looking  at  teachers'  responses,  it  ap- 
pears that  most  of  the  assessment  activities 
tested  from  this  book  require  a high  level 
of  student  involvement.  For  some  teachers 
this  gave  them  ideas  about  how  to  orga- 
nize group  work  and  reminded  them  of 
the  value  of  doing  things  in  groups  and 
having  a balance  of  self  assessment  and 
teacher/facilitator  assessment. 

We  have  highlighted  a few  of  the  re- 
views here.  Since  we  are  not  able  to  reprint 
the  tools  at  this  time  we  chose  those  that 
relied  less  on  seeing  the  tool.  We  looked  at 
a few  so  the  reader  could  get  a flavor  of  the 
kinds  of  activities  the  book  promotes. 

WALK-TALK:  ORAL  TESTS 

I first  asked  the  class  to  talk  about  all  the 
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things  that  they  learned  thus  far  in  our 
three  classes  together.  Second  I had  the 
students  make  up  three  questions  as  if  they 
were  giving  a quiz  or  a test  based  on  the 
materials  they  learned.  I presented  several 
examples  and  told  them  that  true/ false 
questions  could  be  used.  Third,  I paired 
the  students  up  and  told  them  to  go  out- 
side and  ask  questions  of  their  partners. 

For  each  correct  answer  a point  was 
awarded. 

When  I went  out  to  see  how  things  were 
progressing,  I couldn't  help  but  notice  how 
energized  and  upbeat  students  were.  Most 
had  earned  three  points;  all  were  laughing 
and  talking  about  the  activity  and  their  in- 
dividual responses.  I asked  for  feedback 
from  students  as  to  whether  we  should  do 
this  again.  Students  were  enthusiastic. 

In  using  this  assessment  tool  I was  able 
to  see  the  real  value  in  the  students  work- 
ing in  pairs,  in  writing  their  own  ques- 
tions, in  speaking  these  questions  to 
others,  in  having  a good  time  while  ac- 
tively learning,  and  in  receiving  praise 
from  peers  the  further  developed  their  con- 
fidence in  their  overall  communication 
skills. 

— Phyllis  Lee 


MINI  TALK  SHOWS  IN  CLASSROOM 
ASSESSMENT 

In  terms  of  previewing,  I followed  the 
four  suggested  steps  with  one  exception: 
under  number  two,  I replaced  "style  of 
film"  with  "message  or  main  ideas."  I felt 
that  since  the  students  are  not  experts  on 
film  technique,  style  would  most  likely  be 
answered  so  briefly  that  there  would  be 
little  opportunity  for  extended  creative 
speaking. 


879 


Volume  11:  Winter  1998 


Volume 
11  • 83 


View  and  Post  Viewing 

Here  I had  students  organize  their 
thoughts  in  groups  rather  than  as  indi- 
viduals. This  provided  extra  interaction 
as  well  as  opportunities  to  clarify  (debate, 
question,  compare)  their  responses. 

I also  had  them  prepare  questions  to  ask 
other  groups,  with  each  group  responsible 
for  the  questions  on  a specific  area,  e.g.  the 
"character"  group  questioned  the  "mes- 
sage group"  etc.  I felt  that  the  assignment 
of  tasks  and  narrowing  of  focus  would 
facilitate  the  process.  Throughout  these 
discussions,  I circulated,  listened  in  and 
when  invited  intervened. 

Feedback  and  Scoring 

Each  group  presented  on  their  area.  I 
did  not  grade  the  presentations.  The  rest  of 
the  class  provided  each  group  with  sponta- 
neous feedback  on  their  performance. 

Caveats  and  Opinions 

For  the  film  I chose  "Dances  with 
Wolves."  That  was  thematically  related  to 
the  content  of  the  course  "American  Folk- 
lore," specifically  Native  Americans.  The 
45-minute  section  of  the  film  I selected  con- 
centrated on  cross  cultural  relations  and 
prejudices,  the  struggle  to  communicate 
without  a common  spoken  language, 
drawing  upon  other  aids  (visuals)  to  en- 
hance communication,  and  the  gradual 
discovery  of  a common  humanity.  These 
ideas  resonated  greatly  with  the  students' 
lives  and  experiences. 

Overall  both  the  students  and  I found 
this  to  be  an  effective  and  enjoyable  lan- 
guage activity.  They  felt  the  assignment  of 
particular  tasks  (purposeful  watching, 
group  assignments)  helped  with  the  reten- 
tion of  material.  This  was  confirmed  by  ef- 
fusive and  insightful  discussions  following 
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the  movie  and  tasks. 

I allowed  90  minutes  for  the  activity  in- 
cluding 45  for  viewing  time.  The  only  criti- 
cism was  that  this  was  an  insufficient 
amount  of  time  for  their  discussion.  This 
was  a time-consuming  assessment  activity, 
but  I believe  mini  talk  shows  would  be  an 
excellent  assessment  tool  for  general  skills 
or  for  specific  targeted  grammatical  items. 

— Karyn  V.K.  Vitali 

RUN  AND  WIN 

I used  the  Run  and  Win  lesson  with  a 
group  of  16  entry-level  ESOL  students.  It 
was  our  fourth  session  together.  I had  be- 
gun teaching  pronouns  previous  to  the 
suggested  lessons.  Also  I had  demon- 
strated simple  verbs  like  run,  walk,  speak, 
etc.  I used  die  lesson  as  follows: 

Format 

I used  subject  and  verb  cards  to  teach 
subject  verb  agreement. 

Materials 

Cards  made  with  the  pronouns  "I,  He, 
She"  made  up  with  simple  verbs  already 
taught. 

Method 

Since  my  students  had  very  limited 
English  it  would  be  quite  tedious  to  try 
and  explain  the  game  portion  of  the  lesson. 
Instead  I modified  it  by  putting  them  in 
groups  of  5, 5 and  6 at  three  tables.  I 
passed  out  the  verb  cards  which  were  in 
blue  and  white  envelopes.  I instructed 
them  to  open  a blue  card  and  one  white 
card  and  say  the  words  they  see.  I demon- 
strated this  myself  at  each  table.  I moved 
from  table  to  table  allowing  them  to  play 
with  the  verbs  and  continued  to  point  out 
correct  combinations.  Later  they  showed 
me  their  combinations. 
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Conclusion 

The  students  really  enjoyed  the  assess- 
ment. It  gave  them  an  opportunity  to  use 
hands-on  activities  to  make  discoveries  of 
their  own.  They  wanted  more  of  this.  I 
made  up  other  words  that  they  knew  on  a 
purple  card  such  as  "floor,  wall,  book."  I 
also  made  up  yellow  cards  that  say  "in,  on, 
and  at."  Now  they  can  put  together  whole 
sentences  in  this  same  manner  of  discovery. 

I loved  the  assessment  tool.  The  tool  is 
not  only  valuable;  it  is  fun  and  expandable. 

— Jayne  Bissonette 

TENSE  EXCITEMENT  THROWING  A DIE 

The  directions  were  clear.  But,  as  typi- 
cally happens  in  the  classroom,  the  lesson 
did  not  get  presented  according  to  the  di- 
rections. The  copier  at  the  school  did  not 
work,  so  I drew  the  playing  board  on  the 
chalk  board,  and  students  had  to  copy  it. 
We  used  someone  else's  classroom  and 
they  did  not  leave  us  much  on  the  chalk 
board,  so  I couldn't  fit  all  the  words  in  col- 
umns on  the  board.  This  was  fine  because 
the  words  were  too  hard  for  all  but  the  in- 
termediate students. 

So,  we  adapted  the  game  to  suit  us.  We 
did  one  practice  run  using  the  word 
"laugh."  I wrote  it  on  the  board  under  the 
heading  "today."  I then  wrote  it  again  un- 
der the  heading  "yesterday."  This  time  I 
added  the  "-ed"  ending.  We  discussed 
endings  of  the  past  tense  words.  This  was 
actually  an  introduction  for  all  the  begin- 
ning A students.  We  discussed  what  sound 
the  "-ed"  made  in  the  word  "laughed".  We 
all  put  the  word  "laugh"  under  the  "t"  op- 
tion on  the  playing  board  because  we 
make  a "t"  sound  when  we  pronounce  this 
ending. 

We  then  began  the  game.  We  did  not 


roll  the  die  because  students  did  not  have  a 
column  to  choose  from.  I put  a word  on 
the  board  under  "today".  The  students 
said  the  word  and  then  wrote  it  under 
"yesterday".  The  students  tried  to  say  the 
word.  I said  it  correctly  and  repeated  it 
several  times.  With  them  repeating  after 
me,  they  then  chose  the  playing  board 
square  they  thought  the  ending  sound  fit  ( 
t/d/ed).  I went  around  and  checked 
everyone's  answer.  If  they  could  hear  the 
correct  sound.  I also  demonstrated  for 
them  what  the  word  would  sound  like  if  it 
had  incorrect  endings  on  it. 

We  ran  out  of  time,  so  we  only  did  four 
words.  Most  of  the  students  were  putting 
the  ending  sounds  correctly  on  the  playing 
board  by  the  last  two  words.  All  the  stu- 
dents seemed  to  enjoy  the  challenge  of  this 
game.  It  seemed  hard  enough  to  make  it 
fun,  but  not  impossible.  It  seemed  easy  for 
one  intermediate  A student.  Their  interest 
in  the  game  seemed  high.  They  repeated 
the  words  over  to  themselves  to  listen  to 
endings.  They  appeared  pleased  when  I 
came  around  and  told  them  they  had  it 
correct. 

I enjoyed  using  this  assessment  activity. 
It  was  a useful  way  to  introduce  and  prac- 
tice past  tense  ending  sounds.  I think  an 
idea  like  this  is  most  useful  when  any 
teacher  can  apply  it  the  way  it  is  needed 
and  most  useful  in  her  or  his  classroom. 
This  lesson  withstood  this  particular  test.  It 
was  still  an  informative,  learning-filled  ex- 
perience even  when  it  was  changed  a great 
deal  to  fit  within  the  circumstances  in  which 
I was  applying  it.  I will  use  this  lesson  again 
for  past  tense  endings  and  I will  use  the  ba- 
sic game  structure  for  other  sounds  that  I 
want  to  work  on  with  students. 

— Lezlie  S.  Rocka 
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